The first step in knowing whether or not you can use or publish copyrighted material is identifying whether or not the material really is protected by copyright. You can refer to the Introduction to Copyright guide for information on what falls within the public domain and resources for breaking down the duration of copyright protection for different formats created over time.
For some trickier cases, the following resources may help:
Many instructors are comfortable applying fair use to meet educational goals in the classroom, but they may be wary of leveraging this exception for their own scholarly research and publishing activities. Be aware that this exception applies in many cases relevant to scholarship, research, criticism, and commentary, and you can use the same standards and resources provided in the Introduction to Copyright guide as a starting point.
Luckily, many disciplinary and professional communities have also developed guidelines for best practice in fair use, which usually touch on evaluating fair use for the purposes of scholarly presentations or publications. The Center for Media & Social Impact at American University has collaborated on many such guides, available on the Center's website. These guides are not comprehensive, but they offer the perspectives of both domain and legal experts on the issues most relevant to their respective fields. Even if your field is not represented, you may find these guides' recommendations useful, especially if you are using visual or audiovisual materials.
Even if you determine that your use is fair, there might be good reasons to seek permission or to pay licensing fees to use copyrighted material, particularly if you will be widely distributing that material or could potentially profit from its use. Many publishers require authors to clear such rights, and authors are frequently liable for any infringement. This doesn't mean you should shy away from fair use, but you should evaluate your use(s) carefully.
Again, the Introduction to Copyright guide offers some additional guidance and resources on obtaining permission from copyright holders.