The source of supporting materials should be mentioned in the content of the speech and cited in a bibliography at the end of the content (or preparation) outline. Of course, the issue of plagiarism is relevant here, but I prefer to emphasize the need for a speaker to develop credibility. Members of your audience will see you as more believable if they know that your are using recent materials from solid sources.
You should cite sources within the text of your speech, when the information is unique to a source or when the source provides testimony. Decide how much you need to reveal about the source to make it credible (name, sponsoring organization, publication, date), but usually you will not try to cram all this information into your speech. The following is an exception where a number of the elements is important enough to state:
“In an article that appeared in the Duke Law Review in 1990, Nadine Strossen, president of the ACLU, observed that hate speech laws could be written for areas of student privacy like dormitories.”
Because hate speech laws conflict so much with freedom of speech, you want to show that there is a strong argument that they could be constitutional, so you mention the Duke Law Review (a respected publication), 1990 (not that long ago in legal terms), and the president of the ACLU (a strong reference since you would normally expect the ACLU to oppose any law which conflicted with the First Amendment).
Online sites like NoodleTools have free tools for conducting searches for information and creating bibliographies.
In general, supporting materials should be paraphrased when giving a speech. There are several reasons why you should put description, examples, statistics, and testimony into your own words:
Imagine a person was giving a speech on corporal punishment and wanted to use this information:
Psychologist H. Stephen Glenn said "Corporal punishment is the least effective method [of discipline]. Punishment reinforces a failure identity. It reinforces rebellion, resistance, revenge and resentment. And, what people who spank children will learn is that it teaches more about you than it does about them that the whole goal is to crush the child. It's not dignified, and it's not respectful."
Source: Ni, J. K. "Spanking denounced as ineffective, harmful -- Expert at 'Families Alive' [conference] urges positive discipline," Deseret News. 9 May 1998."The American Academy of Pediatrics strongly opposes striking a child. If the spanking is spontaneous, parents should later explain calmly why they did it, the specific behavior that provoked it, and how angry they felt. They might apologize to their child for their loss of control, because that usually helps the youngster understand and accept the spanking."
Source: American Academy of Pediatrics. "Physical punishment." 6 October 2002. <http://www.aap.org/advocacy/childhealthmonth/spank.htm>
In the speech you might say this:
Experts and professional organizations such as the American Academy of Pediatrics tend to frown on corporal punishment. Psychologist H. Stephen Glenn, for example, believes that spanking can strengthen rather than reduce behavior problems.1 Others contend that if a parent spontaneously strikes a child, mom or dad should later discuss their feelings of anger with the child.2
You should note the sources at the bottom of the page or at the end of the outline:
1Ni, 1998
2American Academy of Pediatrics, 2002
In the bibliography, you should list:
Ni, J. K. "Spanking denounced as ineffective, harmful -- Expert at 'Families Alive' [conference] urges positive discipline," Deseret News. 9 May 1998.
American Academy of Pediatrics. "Physical punishment." 6 October 2002. <http://www.aap.org/advocacy/childhealthmonth/spank.htm>