EECS 489: Lab 6

Labs:


	1. Netimg
	2. A Peer Node
	3. ImageDB
	4. DHT O(N) Case
	5. BE Netimg
	6. FEC
	7. TBF
	8. WFQ

PAs:


	1. P2P Search
	2. DHT Search
	3. RDP+FEC
	4. Virtualization

HW:


	HW1
	HW2

Don't see the menu?

EECS 489 Lab 6: Forward Error Correction

This assignment is due on Friday, 25 Mar 2016, 6 pm.

Introduction

This lab have you implement a simple Forward Error Correction (FEC) algorithm on top of the best-effort netimg application from Lab 5. Still assuming the absence of flow control and retransmission, we use a simple XOR-based FEC (or, a linear network code over GF(2)) to correct the loss of a single packet within a given window of data.

Support Code

Before doing anything else, please make a copy of your Lab5 solution. You will need it to start on PA3. In addition to the support files for Lab 5, you're provided with support code consisting of the file fec.cpp and an updated Makefile. You should place these files in your Lab5 solution folder. You're also provided with Linux binary executable of a reference client, reffecimg and reference server, reffecdb in /afs/umich.edu/class/eecs489/w16/lab6/. As with Lab5, these are really netimg and imgdb renamed, to prevent you from running the wrong binaries when testing. The programs still refer to themselves as netimg and imgdb in the diagnostic messages printed to screen. You should be able to run your client against the provided server and the provided client against your server. There are no changes to the command line options of both programs from Lab 5. As usual, you can search for the string "YOUR CODE HERE" in the code to find places where your code goes.

This lab builds upon a working Lab5 solution. If you don't have a working Lab5, you can "purchase" one from the teaching staff at the cost of 15 (ouf of 100) points of your PA3 grades.

Task 1: Server Side

Your first task is to update the server code. You can search for the string "Lab6 Task 1" in imgdb.cpp and fec.cpp to find places where "Task 1" related code must be filled in.

When client sends a query to the server, it also specifies the size of the FEC window (iq_fwnd) to use. The format of the query packet is the same as in Lab5 and is defined in netimg.h. In imgdb::recvqry(), this size is stored in the imgdb::fwnd member variable. The FEC window size is the number of segments from which each FEC data is computed. Think of the FEC window as advancing over the image data, jumping one full window at a time. Each time, prior to an FEC window advance, you must generate an FEC data computed over all the segments within the current FEC window.

Image transmission and FEC computation overview

The image buffer returned by LTGA::GetPixels() may or may not divide evenly into packets of maximum-segment size (mss). You MUST use the buffer AS IS, not to make a copy of the image or resize the buffer prior to transmission. When transmitting an image, all data packets except the last one are of size mss. The image data carried in each packet (the datasize henceforth) is thus mss minus the size of all headers, including the ihdr_t and the UDP and IP headers. As in Lab5, each data byte is assigned a sequence number and the sequence number of a packet, carried in its ihdr_t, is the sequence number of the first data byte in the packet.

One FEC packet is sent for every imgdb::fwnd number of segments. Each FEC packet comprises an ihdr_t header and the FEC data. The FEC data portion of the packet is of a fixed size, i.e., the same datasize as per above and is computed over the image data carried in the most recent FEC window data segments. The sequence number carried in an FEC packet is the sequence number of the first byte of the FEC window over which the FEC data is computed. Only the image data in a packet is used to compute the FEC data; the packet headers, including the ihdr_t header, are not used to compute the FEC data. When an image data packet carries less than datasize amount of image data, the FEC data is updated as if the image data has been padded with 0s up to datasize. The last FEC window of an image may be smaller than imgdb::fwnd (the FEC packet size remains the same fixed size).

Implementation

All changes to the imgdb server code are confined to the function imgdb::sendimg(). You need to figure out how to:

maintain your FEC window,
keep track of your progress over each FEC window, and
compute your FEC data across multiple data segments.

You may want to add member variables to the imgdb class to help you with the above.

Prior to transmitting a data segment, if this is the first segment in an FEC window, use it to initialize your FEC data. Subsequent segments within the FEC window should be XOR-ed with the content of your FEC data. Remember that FEC data is computed over the image data only, not over any headers, not even the ihdr_t header. You MUST use the two helper functions fec.cpp:fec_init() and fec.cpp:fec_accum() to perform your FEC computation for the first and subsequent segments of the FEC window respectively. You are to write these two helper functions.

The last FEC window of an image file may be smaller than imgdb::fwnd size. In which case, just send out the FEC data accumulated so far. The receiver will be smart enough to know how many segments the FEC data has been computed over. Additionally, the last segment in an FEC window may be smaller than datasize. Update your FEC data by XOR-ing it with the content of this last segment; FEC data beyond the size of the last segment, up to datasize, is treated AS IF it had been XOR-ed with 0's. Your fec_init() and fec_accum() MUST be able to handle this corner case. This task should take about 10 lines of code. In addition, the two helper functions each takes about 5 lines of code.

An FEC data packet must be transmitted following the last segment of each FEC-window of data, or the last segment of the image data. The header of the FEC packet MUST have ih_type set to NETIMG_FEC. The sequence number (ih_seqn) of an FEC packet MUST be the sequence number of the first image data byte of the current FEC window. Don't forget to convert all header fields of type integer to network byte order. Call sendmsg() to send your FEC packet. This task takes about 10 lines of code.

That's all for Task 1. It should take about 30 lines of code in total.

Task 2: Client Side

Your second task is to update the client code. You can search for the string "Lab6 Task 2" in netimg.cpp to find places where "Task 2" related code must be filled in. The provided function netimg::sendqry() computes the FEC window size fwnd and sends it to the server in the iq_fwnd field of the iqry_t packet.

The function netimg::recvimg() is where all your work for this task is to be done. You can re-use the struct msghdr to receive both image and FEC data packet. The 5-line of code to do this is an adaptation of your Lab 5 code. As with the server side, you need to figure out how to:

maintain your FEC window,
keep track of your progress over each FEC window, and
compute your FEC data across the multiple data segments to reconstruct a lost packet.

You may want to add member variables to the netimg class to help you with the above.

To reconstruct a lost packet, one stores an FEC-window full of data (minus the lost packet) in a buffer, waits for the FEC data packet to arrive, and computes the lost packet. In our case, we store arriving data directly on the image data buffer, at the correct offset, and perform our FEC computation on the image data buffer itself. Our simple FEC scheme can only reconstruct one lost packet per FEC window. You need to remember the location/offset of the first lost packet within the image buffer (out-of-order packet is considered lost). To avoid the futility of reconstructing a lost packet when multiple packets within an FEC window are lost, you should also keep a count of the total number of packets received per FEC window.

Upon receipt of each data packet, check whether it is the next data packet you're expecting. If not, you've lost a packet, mark the location of the first lost packet in the FEC window. Otherwise, if the packet belongs to the current FEC window, as opposed to a late arriving packet from the previous FEC window, increment your count of total number of packets received within the current FEC window. This task takes 4 lines of code.

Upon receipt of an FEC data packet, check if you've lost only one packet within the FEC window, if so, reconstruct the lost packet. Remember that we're using the image data buffer itself as our FEC window buffer. The sequence number that marks the start of the current FEC window is carried in the ih_seqn field of the FEC data packet. (It may help your debugging to confirm that the sequence number in the arriving FEC packet indeed agrees with what your receiver expects the start of the current FEC window to be.) To reconstruct the lost packet, use fec.cpp:fec_accum() to XOR the received FEC data against the image data buffered, starting from the start of the current FEC window, one datasize at a time, skipping over the lost segment, until you've reached the end of the FEC window. If fec_accum() has been coded correctly, it should handle the case when the last segment of the FEC-window is smaller than datasize.

Once you've reconstructed the lost segment, copy it from the FEC data buffer to the correct offset on the image buffer. If the lost segment is the last segment of the image data, it may be smaller than datasize, in which case, you should copy only the correct amount of bytes. If no packet was lost in the current FEC window, or if more than one packets were lost, there's nothing more you can do with the current FEC window, just move on to the next one. This task takes about 11 lines of code. If multiple packets were lost in an FEC window, the image display will have gaps, this is the correct behavior.

This task is complicated by the following cases: (1) the FEC data packet itself may be lost, and (2) one or more data packets befre and/or after the FEC data packet may also be lost. Thus in addition to relying on fwnd and the count of packets received within an FEC window, you must also check the sequence number in arriving packet to determine when an FEC-window full of data bytes has been sent, instead of relying solely on receipt of FEC packet. Since we do not retransmit lost packet in this lab, if an FEC-window full of data has been sent, even if you haven't received all of it, and the FEC data packet is lost, you simply advance your FEC window one fwnd full. You should then also reset your count of packets received and determine the next expected packet. Be careful that FEC packet could also arrive out of order. A late arriving FEC packet is no longer useful to you if you have already processed the FEC window it corresponds to. In which case, simply throw the FEC packet away without advancing your FEC window again on account of this late arriving FEC packet. This should take about 8 lines of code.

That's it for Task 2. It should take less than 25 lines of code.

Testing Your Code

You should test your code with small-ish mss, say 2048 bytes, and a large enough receiver window size, say 200, to see how FEC can reconstruct lost packets. You know your FEC is working if you see gaps in the displayed image "patched/filled in" out of order. Or you can study the diagnostic messages the reference apps send to stderr. If the output reports one segment dropped in one or more FEC window but the final image is displayed without gaps, your program is working correctly. You should also play with the server's drop probability and observe how higher drop probability causes more gaps/streaks in the displayed image that our simple FEC cannot reconstruct. Multiple lost packets per FEC window will result in gaps in the displayed image. This is the expected/correct behavior. To test your handling of out-of-order packets, run your test over MWireless to increase the probability that some packets will arrive out of order.

Submission Instructions

To incorporate publicly available code in your solution, or to pass off the implementation of an algorithm as that of another are both considered cheating in this course. If you can not implement a required algorithm, you must inform the teaching staff when turning in your assignment.

Your submission must compile and run without errors on CAEN eecs489 hosts using the provided Makefile, unmodified, without any additional libraries or compiler options.

Your "Lab6 files" comprises your fec.cpp, netimg.cpp, imgdb.cpp, and any other support files you modify.

To turn in your Lab6, upload a zipped or gzipped tarball of your Lab6 files to the CTools Drop Box. Keep your own backup copy! The timestamp on your uploaded file is your time of submission. If this is past the deadline, your submission will be considered late. You are allowed multiple "submissions" without late-policy implications as long as you respect the deadline. We highly recommend that you use a private third party repository such as github or M+Box or Dropbox or Google Drive to keep the back up copy of your submission. Local timestamps can be easily altered and cannot be used to establish your files' last modification times (-10 points). Be careful to use only third-party repository that allows for private access. To put your code in publicly accessible third-party repository is an Honor Code violation.

Turn in ONLY the files you have modified. Do not turn in support code we provided that you haven't modified (-4 points). Do not turn in any binary files (object, executable, dll, library, or image files) with your assignment (-4 points). Your code must not require other additional libraries or header files other than the ones listed in the Makefile (-10 points).

Do remove all printf()'s or cout's and cerr's and any other logging statements you've added for debugging purposes. You should debug using a debugger, not with printf()'s. If we can't understand the output of your code, you will get zero point.