For plants
Attach the mirror to the wall similar to what is shown in this diagram for a picture.
For vases and previous glass
OPTION 1: Bees wax or dental wax can restrain small lightweight objects, but can leave a stain on the base of the object. Hook and loop material can also be used, although the glue can also stain. There are also various commercial “putty-like” products available.
OPTION 2: Use mono-filament fishing line for guy wires.