It turns out that you need to add a reference to System.Configuration to the project, rather than simply adding it with a using statement. This can be done as:
Add a reference to System.Configuration by clicking Project -> Add Reference... and searching through the .NET components for System.Configuration