IN4390 Quantitative Evaluation of Embedded Systems: course 2021 - 2022

This course is under construction to swing back to on-campus teaching. Note: contents may be added/removed/outdated at any time.

Instructors

Koen Langendoen Lydia Chen George Iosifidis Marco Zuniga

Course contents

This course offers an introduction into the art of performance evaluation and modeling, with a focus on embedded systems. It covers fundamental tools/techniques like ANOVA, Petri Nets, Markov Chains, and Queueing Theory; and hints at best practices along the way.

The course will involve lectures (twice a week) and a lab (once a week). The course contains customizable contents that counts for 30% towards the final grade as well as compulsory assignments (pass|fail) and a written test that counts for 80% at the end. A detailed explanation of the course setup and grading scheme can be found in this handout.

Schedule and course material

Lecture	Date	Teacher	Topic + slides	Preparation
1	Nov 9	Koen	introduction
2	Nov 11	Lydia	design of experimens	see brighspace
3	Nov 16	Lydia	design of experiments	see brightspace
4	Nov 18	Koen	the art of measuring
5	Nov 23	George	petri nets
6	Nov 25	George	petri nets
7	Dec 2	Marco	markov chains (DT)
8	Dec 7	Koen	operational laws
9	Dec 9	Koen	queueing theory
10	Dec 14	Koen	Q&A QT	practice questions
11	Dec 16	Marco	markov chains (CT)
12	Dec 21	Marco	Q&A MC
13	TBD	Koen++	project presentations

Take home exercises

One of the customization options is to do the following exercises connected to the Design of Experiments (DoE) lectures worth 10 points each.

Excercise Topic Deadline

1 DoE ninja, 10 pts Nov 17

2 DoE DIY, 10 pts Nov 22

Lab

During the lab hours the teaching assistants will be available for Q&A. Take advantage of this service!

Note that assignments 1-3 need to be done in pairs.

Assignment Topic Code Deadline Customizables

0 see brighspace Nov 15

1 ROS 2 lab-1 Dec 6 ≤ 120 pts, Dec 17

2 Petri nets Dec 13 ≤ 60 pts, Dec 23

3 Putting it together lab-3 Jan 17 ≤ 100 pts, Jan 24

Projects

To be done in pairs.

Project list, note: proposals accepted FCFS

Reading list

Basic Concepts and Taxonomy of Dependable and Secure Computing, A. Avizienis ; J.-C. Laprie ; B. Randell ; C. Landwehr
IEEE Transactions on Dependable and Secure Computing (Volume: 1 , Issue: 1 , Jan.-March 2004)
The Art of Computer Systems Performance Analysis, R. Jain
Wiley and Sons, 1991
Chapters 18-20,22-23 (DoE), and 30-33 (Queueing Theory)
Petri nets: Properties, analysis and applications, T. Murata
Proceedings of the IEEE, vol. 77, no. 4, pp. 541-580, April 1989
Introduction to probability, statistics, and random processes, H. Pishro-Nik
Kappa Research LLC, 2014
Chapters 11.2 (DTMC), and 11.3 (CTMC)

Supplementary list

Performance Analysis of the IEEE 802.11 Distributed Coordination Function, G. Bianchi
IEEE Journal on Selected Areas in Communications, vol. 18, no. 3, pp. 535-547, March 2000

Exams

Exam	Answers
practice exam	answers
2020-01-30	answers
2020-04-03	answers
2021-01-25	answers
2022-01-24	answers

Contact

For questions regarding the labs please contact one of the teaching assistants:

Naram Mhaisen (N.Mhaisen@tudelft.nl)

All other issues can be discussed with the principal lecturer:

Koen Langendoen (k.g.langendoen@tudelft.nl)

Excercise	Topic	Deadline
1	DoE ninja, 10 pts	Nov 17
2	DoE DIY, 10 pts	Nov 22

Assignment	Topic	Code	Deadline	Customizables
0	see brighspace		Nov 15
1	ROS 2	lab-1	Dec 6	≤ 120 pts, Dec 17
2	Petri nets		Dec 13	≤ 60 pts, Dec 23
3	Putting it together	lab-3	Jan 17	≤ 100 pts, Jan 24