If you are saying that when you press play to start the video, the video starts playing on the Attendee's computer seconds or even minutes later, then that has been normal for me as well. In some cases, the video starts playing on the Attendee's computer before it starts playing on the Organizer's computer.
The video is streamed directly from the GoTo cloud directly to each computer. Many factors affect who gets to see the video first. The computer operating system, other programs running on the computer at the same time, the strength of the wifi signal if not connected direct to a network, download speed of the Internet connection, and other network traffic if on a shared network all play a hand in how quickly the video starts playing on each device.
Other factors include total Internet traffic, especially traffic in your same time zone. In the Eastern US we are fortunate that the Internet is fairly quiet in the early morning since there are not many people using the Internet in the time zones that span the Atlantic Ocean. Later in the afternoon the Internet can be quite busy.
I have also found that the particular encoding for your video can make a difference in download time. Search these discussion boards for tips on creating a "compatible" mp4 file that should work best on most devices. There are thousands of possible ways to encode an MP4 video, and some work better than others. A 540p video will have less data transfer across the Internet than a 720p video. If you don't need the resolution of a 720p (or higher) then try lowering the resolution for more compatibility.