15 people were also injured during the incident
The incident began when the driver mowed down two pedestrians, killing one person. While trying to speed away he ran over a group of musicians.
Officials have put the death toll at 34, with another 15 people said to be injured.
The accident happened about 90 miles outside of the capital Port-au-Prince, early on Sunday.
Speaking about the crash, Faustin Joseph, civic protection coordinator for the area said "The people who were not victims of the accident tried to burn the bus with the passengers inside."
"The bus, the passengers and the driver were all placed into the care of the local authorities."