All class meetings with the instructor will take place on Zoom.
Office hours will start from the second week of the course. The instructor's office hours are group office hours and will be on Zoom. The instructor's office hours will be recorded. The TA's office hours are individual and will be on Zoom.
| Week | Date | Type | Topic |
| 1 | - | - | - |
| 01/20/2021 | Class | Introduction | |
| 2 |
01/25/2021 | Class | Software Verification and Validation |
| 01/27/2021 | Class | Software Systems | |
| 01/28/2021 | Office Hours | - | |
| 3 |
02/01/2021 | Class | Building Software Systems |
| 02/03/2021 | Class | Team Building Activity | |
| 02/04/2021 | Office Hours | - | |
| 4 |
02/08/2021 | Class | Unit testing |
| 02/10/2021 | Class | Test Doubles | |
| 02/11/2021 | Office Hours | - | |
| 5 |
02/15/2021 | Class | System Testing |
| 02/17/2021 | Class | Acceptance Testing + CI | |
| 02/18/2021 | Office Hours | - | |
| 6 |
02/22/2021 | Class | Program Representations I |
| 02/24/2021 | Class | Program Representations II | |
| 02/25/2021 | Office Hours | - | |
| 7 |
03/01/2021 | Class | Control-flow Coverage |
| 03/03/2021 | Class | Mutation Coverage | |
| 03/04/2021 | Office Hours | - | |
| 03/05/2021 | Office Hours |
- |
|
| 8 |
03/08/2021 | Class | Data-flow Coverage |
| 03/10/2021 | Class | Random Testing | |
| 03/11/2021 | Office Hours | - | |
| 9 |
03/15/2021 | Class | Partition Testing |
| 03/15/2021 | Office Hours |
- |
|
| 03/17/2021 | Class | Combinatorial Testing | |
| 03/18/2021 | Office Hours | - | |
| 10 |
03/22/2021 | Class | Model-based Testing |
| 03/22/2021 | Office Hours | - |
|
| 03/24/2021 | Class | Search-based Testing | |
| 03/25/2021 | Office Hours | - | |
| 11 |
03/29/2021 | Class | Symbolic Execution |
| 03/29/2021 | Office Hours | ||
| 03/31/2021 | Class | Fuzzing | |
| 04/01/2021 | Office Hours | - | |
| 12 |
04/05/2021 | - | - |
| 04/05/2021 | - | - |
|
| 04/07/2021 | - | - | |
| 04/08/2021 | - | - | |
| 13 |
04/12/2021 | Class | Test Input Minimization |
| 04/12/2021 | Office Hours | - |
|
| 04/14/2021 | Class | Flaky Tests |
|
| 04/15/2021 | Office Hours | - | |
| 14 |
04/19/2021 | Class | Regression Testing I |
| 04/19/2021 | Office Hours | - |
|
| 04/21/2021 | Class | Regression Testing II | |
| 04/22/2021 | Office Hours | - | |
| 15 |
04/26/2021 | Class | Testing Tech Talks |
| 04/26/2021 | Office Hours |
- | |
| 04/28/2021 | Class | Testing Tech Talks | |
| 04/29/2021 | Office Hours | - | |
| 16 | 05/03/2021 | Class | Testing Tech Talks |
| 05/03/2021 | Social Hour |
- |
You should refer to this course site and our Piazza site for finding information about course meetings, assessments, and grades. We will be meeting on Zoom for both classes and office hours. For class meetings, we will provide video recordings. We will use Piazza for all communications between students, the instructor, and the course staff. (The course staff will add you to Piazza using your UMN email.) We use Piazza as it is a great place to promote interactions among students, and the course staff can keep track of all course-related discussions in one place. You will be using Gradescope to complete participation assessments and quizzes. You will be using GitHub, GitHub classroom, and Gradescope to complete coding assignments. You will use Gradescope to submit testing tech talks.
You can find all the assessment tasks on the Assignments page of this Canvas site and on Gradescope. Please check this page at the end of each course meeting for updated information.
Final course grades will be calculated based on the following percentages:
| Percentage | Assessment Type | Submission Type |
| 10% | Participation | Individual |
| 40% | Quizzes | Individual |
| 40% | Coding Assignments | Team |
| 10% | Testing Tech Talks | Team |
Participation: Due to the fact that all of us will not be in the same time zone, attending synchronous Zoom meetings is not required but highly encouraged. The participation grade will be based on offline assessments that include questions already answered in the class meetings. Specifically, at the end of each class meeting, you will need to complete a participation assessment on Gradescope. The assessment will be open for 48 hours. For people that cannot attend course meetings synchronously, we will provide meeting recordings, and link the recordings in the Topics table above. Participation assessments are individual.
Quizzes: There will be six quizzes. However, the quiz with the lowest score will be automatically dropped from the final grade computation. Quizzes will be available for 48 hours after they are released but once you start them they will be open for a limited amount of time. Quizzes need to be performed individually.
Coding Assignments: There will be six coding assignments. Coding assignments are based on teams of two students. In the coding assignments, your team will use technologies that implement the theoretical concepts discussed in the course. We will use peer evaluations to help us assess how much the individual team members contributed to the results of the team in the assignments. Please note that we will (1) give a lower weight to outliers (e.g., discount one negative rating when all other ratings for that student are positive), (2) consider individual cases (e.g., specific problems that might have affected the performance), and (3) provide the opportunity to detail the individual contributions for each of team assessments. In general, we will use a grain of salt when taking the ratings into account.
Testing Tech Talks: Your team will create a video discussing a testing technology. This video will be played to the class in one of the last three lectures. The testing tech talk is a team-based assessment. We will use peer evaluations also for tech talks.
Exams: There is no midterm or final exam.
Late work is not accepted in this class. There are a few reasons for this policy. First, the assessment schedule follows a tight timeline and we would like to grade assessments as quickly as possible. Second, you will have ample time to complete the assessments. Finally, the majority of the assessments will help you complete immediately following assessments.
Requests for items to be regraded must be made within seven calendar days of the marks being posted. Be aware, this means that students may not ask for an assignment from early in the term to be regraded after they receive their final grade. If a mistake has been made in recording a student's marks, please bring this to the attention of the course staff prior to the end of the term.
Final grades will be assigned based on the following scale. Grading for this course is on an absolute scale (i.e. no curve, no rounding up, etc.).
| Weighted Score (x) | Letter Grade | S/N |
| 94.0% ≤ x ≤ 100.0% | A | S |
| 90.0% ≤ x < 94.0% | A- | S |
| 87.0% ≤ x < 90.0% | B+ | S |
| 83.0% ≤ x < 87.0% | B | S |
| 80.0% ≤ x < 83.0% | B- | S |
| 77.0% ≤ x < 80.0% | C+ | S |
| 73.0% ≤ x < 77.0% | C | S |
| 70.0% ≤ x < 73.0% | C- | S |
| 65.0% ≤ x < 70.00 | D+ | N |
| 60.0% ≤ x < 65.0% | D | N |
| 0% ≤ x < 60.0% | F | N |
Due to the current circumstances, we will also process requests for an incomplete "I" grade. The I grade indicates that the instructor has (1) reasonable expectations that the student can complete an unfinished course on their own no later than the end of the next semester and (2) believes that legitimate reasons exist to justify extending the deadline for course completion. The only acceptable reasons will be documented illness or personal emergency. A written explanation (including supporting documentation) must be submitted to your instructor; if the explanation is acceptable, an Agreement for the Completion of Incomplete Work will be filled out as a contract between the student and the instructor.
I would like to also share some of my thoughts on final grades. First and foremost, your grade isn't a judgment of who you are as a person. It is not an overarching statement about your fitness for work in this major or this field. It is not even necessarily a statement about how much of the course material you know or how hard you tried. It is a summary of the record of how you did on the required assessments for this course.
Second, if you mention to me that you think that your grade doesn't reflect your effort or your understanding, I want to let you know that I believe that. I want to let you know that you are so much more than that number or letter on a page. But if you ask me whether it is possible to change your grade, please understand that you are asking me to falsify the record of how you did on the required assessments for this course. I cannot do that. I can tell you that even if your grade is not what you would have hoped it would be, I still believe in you and in the dream behind that hope. I 100% believe that you can go on to be successful in this major and in the field throughout your life.
CSCI5801 is the formal prerequisites for this class. You must be accepted to the CS upper-division, be a CS graduate student, or receive department permission to take this class. You are expected to have previous programming experience in Java, know how to use Unix-like command-line tools (e.g., cp), and be familiar with Git. You are expected to know basic data structures (such as lists), algorithms (such as search), recursion, and data abstraction. You are expected to be proficient in English and using a computer to produce text documents and figures as required by the assignments. Please contact the instructor on Piazza if you have any questions about whether the course is a good fit for your background and academic goals.
In this course, you will learn the theory, skills, and tools to test real-world software systems. This course will prepare you for a career in software engineering. The topics covered in the course include:
In this course, you will:
This course covers a medium-large amount of material, requires good programming skills, and needs a significant amount of time spent on using software testing tools. Therefore the course will require a good amount of time to do all the reading, programming, and studying for quizzes. Students should expect to spend an average of 9 to 12 hours a week on this course.
In previous courses, you’ve were introduced to software testing. In this course, you will learn software testing techniques used in industry and learn how to use the tools that implement these techniques. In the course, you will be given access to a real-world software system and you will use the system in the coding assignments of the course.
The first priority of the course staff (i.e. TA) is to ensure that each student receives the support they need to understand the course content. Please see them first with any questions about the course content. The course staff also forms the first line of support when it comes to the assignments.
There is no required textbook and the course meetings together with the course's slides will help you prepare for the assessments. There are a few books that we suggest if you would like to further explore some of the concepts we will cover throughout the course. The books are the followings:
All work submitted for this course is required to be your original work or that of your group in the case of group work. You are expected to do your own thinking about how to solve an assessment. You are encouraged to discuss the content of the lectures and the texts with your peers. If you have any questions about whether discussing something with peers might go beyond what is permitted, then stop and ask us first on Piazza for clarification on the policy.
You are also not allowed to post your solutions to a repo that is not the one we assigned to you. We will also use code plagiarism detectors in this class.
The web is a fantastic resource for learning about testing, but it is also a potential source for solutions to assignments, so it is important that we are all on the same page with regard to what is permitted use of an online resource for this class and what is not. The use of the web in a way that supplements the type of information you will find in your class materials is great. This is encouraged. For example, if we talk about dynamic symbolic execution in class, you are encouraged to search for "dynamic symbolic execution" online and read more about this as a generic technique that can be used to test many programs. However, when it comes to a question in a quiz, you are not allowed to search online for something like "what is the result of symbolically executing if (x==3)". This crosses a line into searching out the solution to an assessment rather than learning about dynamic symbolic execution.
Scholastic dishonesty includes any deceptive means whereby a student attempts to gain an unfair advantage. Examples of scholastic dishonesty include violating the course policies outlined here, especially its “Academic Integrity” section; plagiarizing; cheating on assignments; or engaging in unauthorized collaboration on academic work, either with other students or via the internet. In order to be as clear as possible about your scholastic conduct responsibilities and how these relate specifically to the types of courses that we teach in the Department of Computer Science & Engineering, the faculty have prepared a CS&E Department Academic Conduct Policy. Our course will follow this policy, which stands alongside the broader Board of Regents Student Conduct Code.
Within the course, a student responsible for scholastic dishonesty can be given a penalty, including an "F" or "N" for the course. I am also required to report the incident to the Office for Student Conduct and Academic Integrity, and further disciplinary action may occur.
You are responsible for knowing and following the policies on scholastic conduct that are described in the syllabus and in the related documents discussed above (see especially the CS&E Department Academic Conduct Policy).
University policy is to provide, on a flexible and individualized basis, reasonable accommodations to students who have documented disability conditions (e.g., physical, learning, psychiatric, vision, hearing, or systemic) that may affect their ability to participate in course activities or to meet course requirements. Students with disabilities are encouraged to contact Disability Services and their instructors to discuss individual needs for accommodations. Disability Services, McNamara Alumni Center, Suite 180, 200 Oak Street, East Bank. Staff can be reached by calling (612) 626-1333 (voice or TTY).
As a student you may experience a range of issues that can cause barriers to learning, such as strained relationships, increased anxiety, alcohol/drug problems, feeling down, difficulty concentrating, and/or lack of motivation. These mental health concerns or stressful events may lead to diminished academic performance or reduce your ability to participate in daily activities. University of Minnesota services are available to assist you with addressing these and other concerns you may be experiencing. You can learn more about the broad range of confidential mental health services available on campus here.
In this course, we are committed to the University's equal access and opportunity policy. The University of Minnesota shall provide equal access to and opportunity in its programs, facilities, and employment without regard to race, color, creed, religion, national origin, gender, age, marital status, disability, public assistance status, veteran status, sexual orientation, gender identity, or gender expression.
In this course, we are committed to the University's Sexual Harassment, Sexual Assault, Stalking and Relationship Violence policy. The University of Minnesota is committed to taking prompt and effective steps intended to end sexual harassment, sexual assault, stalking, relationship violence, and related retaliation, prevent their recurrence, and, as appropriate, remedy their effects.
Zoom: We will use Zoom for class and office hours meetings. To allow all students to follow the course, we will record the meetings, use Kaltura to generate subtitles, and provide the recordings on this site. (We will not make the recordings publicly available.) If you have concerns about your visibility on Zoom (e.g., your video on Zoom), you are free to turn off your video during the meetings, and you should not be visible in the recording. If you have other concerns, please contact the instructor on Piazza for further information.
Websites: In this course, our use of technology will involve sharing students' names, UMN usernames, and/or coursework information to course-related websites (e.g., Gradescope). All web sites might be affected by security vulnerabilities and it could be possible for them to be the target of software attacks. If you have concerns about the visibility of your name, username, or other information, please contact the instructor on Piazza for further information. Rest assured that it is our priority to keep this information within the boundaries of these websites.
All students are expected to behave as scholars at a leading institute of technology. This includes raising "virtual" hands during Zoom meetings to ask questions and not talking over each other during the meetings. Disruptive students will be warned and potentially dismissed from the meetings.
We reserve the right to make changes to the syllabus to accommodate for exceptional circumstances that are out of the course staff's control. We will announce any change if it needs to happen.