Small chucks are usually way out of line if you dont spend the money for a quality item, collets are for that purpose, to run the cutters perfectally true (although even some very cheap collets dont run true!!) a chuck is totally fine in other uses but on a dremel running as fast as it does it could damage the bearings and actually cause any bits to break up sooner and cause the tool to be more dangourus as it is anyway in the wrong hands,
That said, it is a very good idea and one i often use in other applications, if you can make sure your chuck is on alignment be it with a dti or just line of sight and there is no vibration when the unit is switched on then go for it,
Not scare mongering just making sure you all know the pros and cons,