EECS 489 Lab 6: Forward Error Correction
This assignment is due on Friday,
25 Mar 2016, 6 pm.
Introduction
This lab have you implement a simple Forward Error Correction (FEC)
algorithm on top of the best-effort netimg application from
Lab 5. Still assuming the absence of flow control and retransmission,
we use a simple XOR-based FEC (or, a linear network code over GF(2)) to
correct the loss of a single packet within a given window of
data.
Support Code
Before doing anything else, please make a copy of your
Lab5 solution. You will need it to start on PA3. In addition
to the support files for Lab 5, you're provided with
support code consisting of the file fec.cpp and an
updated Makefile. You should place these files in your Lab5
solution folder. You're also provided with Linux binary executable of
a reference client, reffecimg and reference server,
reffecdb in
/afs/umich.edu/class/eecs489/w16/lab6/. As with
Lab5, these are really netimg and imgdb renamed, to
prevent you from running the wrong binaries when testing. The
programs still refer to themselves as netimg and
imgdb in the diagnostic messages printed to screen. You
should be able to run your client against the provided server and the
provided client against your server. There are no changes to the
command line options of both programs from Lab 5. As usual, you can
search for the string "YOUR CODE HERE" in the code to find places
where your code goes.
This lab builds upon a working Lab5 solution. If you don't have a
working Lab5, you can "purchase" one from the teaching staff at the
cost of 15 (ouf of 100) points of your PA3 grades.
Task 1: Server Side
Your first task is to update the server code. You can search for the
string "Lab6 Task 1" in imgdb.cpp and fec.cpp to
find places where "Task 1" related code must be filled in.
When
client sends a query to the server, it also specifies the size of the
FEC window (iq_fwnd) to use. The format of the query
packet is the same as in Lab5 and is defined in netimg.h. In
imgdb::recvqry(), this size is stored in the
imgdb::fwnd member variable. The FEC window size is
the number of segments from which each FEC data is computed. Think of
the FEC window as advancing over the image data, jumping one full
window at a time. Each time, prior to an FEC window advance, you must
generate an FEC data computed over all the segments within the
current FEC window.
Image transmission and FEC computation overview
The image buffer returned by LTGA::GetPixels() may or may not
divide evenly into packets of maximum-segment size (mss).
You MUST use the buffer AS IS, not to make a copy of the
image or resize the buffer prior to transmission. When transmitting
an image, all data packets except the last one are of size
mss. The image data carried in each packet (the
datasize henceforth) is thus mss minus the size of
all headers, including the ihdr_t and the UDP and IP headers.
As in Lab5, each data byte is assigned a sequence number and the
sequence number of a packet, carried in its ihdr_t, is the
sequence number of the first data byte in the packet.
One FEC packet is sent for every imgdb::fwnd number of
segments. Each FEC packet comprises an ihdr_t header
and the FEC data.
The FEC data portion of the packet is of a fixed size, i.e., the
same datasize as per above and is computed over the image
data carried in the most recent FEC window data segments.
The sequence number carried in an FEC packet is the sequence
number of the first byte of the FEC window over which the FEC data is
computed. Only the image data in a packet is used to compute the FEC
data; the packet headers, including the ihdr_t header, are
not used to compute the FEC data. When an image data packet
carries less than datasize amount of image data, the FEC data
is updated as if the image data has been padded with 0s up to
datasize. The last FEC window of an image may be smaller than
imgdb::fwnd (the FEC packet size remains the same fixed size).
Implementation
All changes to the imgdb server code
are confined to the function imgdb::sendimg(). You need to
figure out how to:
- maintain your FEC window,
- keep track of your progress over each FEC window, and
- compute your FEC data across multiple data segments.
You may want to add member variables to the imgdb class to help you
with the above.
Prior to transmitting a data segment, if this is the first segment in
an FEC window, use it to initialize your FEC data. Subsequent
segments within the FEC window should be XOR-ed with the content of
your FEC data. Remember that FEC data is computed over the image data
only, not over any headers, not even the ihdr_t header.
You MUST use the two
helper functions fec.cpp:fec_init() and
fec.cpp:fec_accum() to perform your FEC computation for the
first and subsequent segments of the FEC window respectively. You are
to write these two helper functions.
The last FEC window of an image file may be smaller than imgdb::fwnd
size. In which case, just send out the FEC data accumulated so far. The
receiver will be smart enough to know how many segments the FEC data has
been computed over.
Additionally, the last segment in an FEC window may be smaller than
datasize. Update your FEC data by XOR-ing it with the content
of this last segment; FEC data beyond the size of the last segment, up
to datasize, is treated AS IF it had been XOR-ed with 0's. Your
fec_init() and fec_accum() MUST be able to handle
this corner case. This task should take about 10 lines of code. In
addition, the two helper functions each takes about 5 lines of code.
An FEC data packet must be transmitted following the last segment of
each FEC-window of data, or the last segment of the image data. The
header of the FEC packet MUST have ih_type set to
NETIMG_FEC. The sequence number (ih_seqn) of an FEC
packet MUST be the sequence number of the first image data byte of the
current FEC window. Don't forget to convert all header fields of type
integer to network byte order. Call sendmsg() to send your
FEC packet. This task takes about 10 lines of code.
That's all for Task 1. It should take about 30 lines of code in
total.
Task 2: Client Side
Your second task is to update the client code. You can search for the
string "Lab6 Task 2" in netimg.cpp to find places where "Task
2" related code must be filled in. The provided function
netimg::sendqry() computes the FEC window size fwnd
and sends it to the server in the iq_fwnd field of the
iqry_t packet.
The function netimg::recvimg() is where all your work for this
task is to be done. You can re-use the struct msghdr to
receive both image and FEC data packet. The 5-line of code to do this
is an adaptation of your Lab 5 code. As with the server side, you
need to figure out how to:
- maintain your FEC window,
- keep track of your progress over each FEC window, and
- compute your FEC data across the multiple data segments to reconstruct a lost packet.
You may want to add member variables to the netimg class to help you
with the above.
To reconstruct a lost packet, one stores an FEC-window full of data
(minus the lost packet) in a buffer, waits for the FEC data packet to
arrive, and computes the lost packet. In our case, we store arriving
data directly on the image data buffer, at the correct offset, and
perform our FEC computation on the image data buffer itself.
Our simple FEC scheme can only reconstruct one lost
packet per FEC window. You need to remember the location/offset of
the first lost packet within the image buffer (out-of-order packet is
considered lost). To avoid the futility of reconstructing a lost
packet when multiple packets within an FEC window are lost, you should
also keep a count of the total number of packets received per FEC
window.
Upon receipt of each data packet, check whether it is the next data
packet you're expecting. If not, you've lost a packet, mark the
location of the first lost packet in the FEC window. Otherwise, if
the packet belongs to the current FEC window, as opposed to a late
arriving packet from the previous FEC window, increment your count of
total number of packets received within the current FEC window. This
task takes 4 lines of code.
Upon receipt of an FEC data packet, check if you've lost only one
packet within the FEC window, if so, reconstruct the lost packet.
Remember that we're using the image data buffer itself as our FEC
window buffer. The sequence number that marks the start of the
current FEC window is carried in the ih_seqn field of the FEC
data packet. (It may help your debugging to confirm that the sequence
number in the arriving FEC packet indeed agrees with what your
receiver expects the start of the current FEC window to be.) To
reconstruct the lost packet, use fec.cpp:fec_accum() to XOR
the received FEC data against the image data buffered, starting from
the start of the current FEC window, one datasize at a time,
skipping over the lost segment, until you've reached the end of the
FEC window. If fec_accum() has been coded correctly, it
should handle the case when the last segment of the FEC-window is
smaller than datasize.
Once you've reconstructed the lost segment, copy it from the FEC data
buffer to the correct offset on the image buffer. If the lost segment
is the last segment of the image data, it may be smaller than
datasize, in which case, you should copy only the correct
amount of bytes. If no packet was lost in the current FEC window, or
if more than one packets were lost, there's nothing more you can do
with the current FEC window, just move on to the next one. This task
takes about 11 lines of code. If multiple packets were lost in an FEC
window, the image display will have gaps, this is the correct
behavior.
This task is complicated by the following cases: (1) the FEC data
packet itself may be lost, and (2) one or more data packets befre
and/or after the FEC data packet may also be lost. Thus in addition
to relying on fwnd and the count of packets received within
an FEC window, you must also check the sequence number in arriving
packet to determine when an FEC-window full of data bytes has been
sent, instead of relying solely on receipt of FEC packet. Since we do
not retransmit lost packet in this lab, if an FEC-window full of data
has been sent, even if you haven't received all of it, and the FEC
data packet is lost, you simply advance your FEC window one
fwnd full. You should then also reset your count of packets
received and determine the next expected packet. Be careful that FEC
packet could also arrive out of order. A late arriving FEC packet is
no longer useful to you if you have already processed the FEC
window it corresponds to. In which case, simply throw the FEC packet
away without advancing your FEC window again on account of
this late arriving FEC packet. This should take about 8 lines of
code.
That's it for Task 2. It should take less than 25 lines of code.
Testing Your Code
You should test your code with small-ish mss, say 2048 bytes, and a
large enough receiver window size, say 200, to see how FEC can
reconstruct lost packets. You know your FEC is working if you see gaps
in the displayed image "patched/filled in" out of order. Or you can
study the diagnostic messages the reference apps send to
stderr. If the output reports one segment dropped in one or
more FEC window but the final image is displayed without gaps, your
program is working correctly. You should also play with the server's
drop probability and observe how higher drop probability causes more
gaps/streaks in the displayed image that our simple FEC cannot
reconstruct. Multiple lost packets per FEC window will result in gaps
in the displayed image. This is the expected/correct behavior. To test
your handling of out-of-order packets, run your test over MWireless to
increase the probability that some packets will arrive out of order.
Submission Instructions
To incorporate publicly available code in your solution, or to pass
off the implementation of an algorithm as that of another are both
considered cheating in this course. If you can not implement a
required algorithm, you must inform the teaching staff when turning in
your assignment.
Your submission must compile and run without errors on CAEN
eecs489 hosts using the provided Makefile, unmodified, without any additional libraries or
compiler options.
Your "Lab6 files" comprises your fec.cpp,
netimg.cpp, imgdb.cpp, and any other support files
you modify.
To turn in your Lab6, upload a zipped
or gzipped
tarball of your Lab6 files to the CTools Drop Box. Keep your own backup copy! The timestamp on your
uploaded file is your time of submission. If this is past the
deadline, your submission will be considered late. You are allowed
multiple "submissions" without late-policy implications as
long as you respect the deadline. We highly recommend that you use a
private third party repository such as github
or M+Box or Dropbox or Google Drive to keep the back up copy of your
submission. Local timestamps can be easily altered and cannot be used
to establish your files' last modification times (-10 points). Be
careful to use only third-party repository that allows for
private access. To put your code in publicly accessible
third-party repository is an Honor Code
violation.
Turn in ONLY the files you have modified. Do
not turn in support code we provided that you haven't modified (-4 points).
Do not turn in any binary files (object, executable, dll,
library, or image files) with your assignment (-4 points). Your code
must not require other additional libraries or header files other
than the ones listed in the Makefile (-10 points).
Do remove all printf()'s or
cout's and cerr's and any other logging statements
you've added for debugging purposes. You should debug using a
debugger, not with printf()'s. If we can't understand the
output of your code, you will get zero point.