Plastic bags don't litter - people do:
Litter continues to be a problem on our roadsides, countryside and seashores. The cost of cleaning up is considerable: it costs nine times more to clean litter from the roads than collecting it from households.
Litter also has a negative effect on tourism by making our country appear dirty and uncared for. Post-consumer waste of all types is a major environmental problem. The same properties that make plastics so useful to man - low mass and strength ratio and durability - also cause plastics litter to be highly visible in the environment. Other forms of litter, such as paper, broken glass and rusty metal are often less obvious but more dangerous than plastics litter. Litter basically results from irresponsible social attitudes, which can only be changed by education and awareness programmes.
Plastic bags are not toxic or harmful to health:
Plastic bags are basically not toxic and are considered unreactive in normal use. During the course of manufacture, no toxic discharges are generated. Even if discarded after use, plastic bags have been proven not to taint or form toxic substances that will affect groundwater or the atmosphere if recycled or put into landfills.
(Reference: It's all about plastics - material of choice)