To practice the material taught in the lectures you should attempt exercises here for Markov chains and here for Markov decision processes.
This hands-on tool session will introduce you to PRISM. To get started, download and install it. It is best to make sure that you are running the latest version (currently 4.3), but any version from 4.1 onwards should be fine.
This lab session is self-contained, but if you want information about running PRISM the best source is the online manual.
There are three parts, of which Parts A and B are of standard difficulty, and Part C is considered advanced:
If you want to learn about other aspects of PRISM, take a look at the main online tutorial.
You can also find many more interesting examples in the case studies section of the website.