When I went to Victoria I took the BC Ferries Connector. Bus pick up in Vancouver, ferry, then the same bus and it dropped me in downtown Victoria. Super easy.Yup. As blabbermouth mentioned.
One big warning though: the ferry terminal is not in Victoria proper, and neither is the terminal in the Vancouver area (It is actually Twassessen). But there are coach buses, local transit buses, and various car services which can get you to these terminals.