Coverity assignment

In this assignment, we will use Coverity to search for bugs in a high-profile open source project. Hopefully we will find some!

For this assignment it is recommended to work together in a team of two or three people. Also, since you can see (in Coverity Connect) what other groups are doing, you are also allowed to collaborate and communicate with other groups.

You will be graded on the basis of a report. You can submit this report in Brightspace; the deadline for submission is Tuesday 30 April 2019, 23:59.

You can submit this report as a PDF file or as a plain-text report.txt.

Don't forget to mention the student id of you and your partner(s). If you exchanged information with other groups, mention this (give credit where credit is due!) and explain how you used this in your search for defects.

Task: find bugs in curl

The program curl probably does not need introduction. Your task is very simple: put Coverity through its paces by using it to analyze curl.

Note that this will be a challenging assignment, as the maintainers of curl are aware of Coverity and have put it through the open-source scan service, as you can see here.

However, since we have a license for the proprietary version, you will be able to dig a bit deeper. As a side-benefit of this assignment, an open source program will become a little bit better by having been scrutinized by many eyeballs.

Instructions

1. Clone the latest version of curl from the Github repository, and figure out how to build it.

2. Apply Coverity Scan on curl (again, you can follow these instructions). Initially, it is recommended to use cov-analyze using its default aggressiveness level (low), otherwise you will get a lot of defects, many of which might be false positives.

3. Analyse the list of defects. There will probably be too many to go through all of them in detail, so you will have to decide what defects you want to focus on. I.e. focus on defects that you think are likely real issues.

4. If you have found a real issue, analyze how it could lead to problems and under what scenario. If possible, fix the issue in the curl source so that the underlying problem (and the Coverity warning) goes away.

   Remember that Coverity is neither sound nor complete, and that blindly following the advice of a static analysis tool can also be harfmul.

5. (Optional ) If you are convinced of your result, communicate this to the authors of curl by opening an issue on the GitHub page, or perhaps even proposing a fix and initiating a pull request.

Questions

1. Describe the steps you took in analyzing curl. How did you build and analyze it, and how did you decide what defect reports to focus on?

2. Describe the result of your triage.

   • If you (only) classified defects as false positives or intentional -- describe a few of the more interesting cases, and why you think they are false positives/intentional.
   • If you found a real issue, describe your analysis from step 4 above in the form of a bug report.
   • How did you collaborate/coordinate your bug-finding efforts with other groups?

3. Write a short evaluation in which you give your opinion about the strengths/weaknesses of Coverity. How would you rate its usefulness compared with a more formal verification tool such as Why3?