Phil goes in-depth here.
"I just talked to Buzz Aldrin on the phone, and he notes that the quotations were taken out of context and did not convey the intended meaning. After the Apollo 11 crew verified that the object they were seeing was not the SIVB upper stage, which was about 6000 miles away at that time, they concluded that they were probably seeing one of the panels from the separation of the spacecraft from the upper stage. These panels were not tracked from Earth and were likely much closer to the Apollo spacecraft. They chose not to discuss this on the open communications channel since they were concerned that their comments might be misinterpreted (as they are being now). Apparently all of this discussion about the panels was cut from the broadcast interview, thus giving the impression that they had seen a UFO." -Dave Morrison