EECS 489 PA2: DHT Search

When a node first forms or joins a DHT, set all its fingers to point to itself. When a node accepts a joining node, it makes the joining node its predecessor as in Lab 4. Similarly, the joining node gets as its successor the node accepting its join request; and its predecessor is the accepting node's old predecessor. (See the discussion on fixup() and fixdn() below for how the rest of the finger table is updated.) When forwarding a join packet, instead of simply forwarding to the successor node, a node first finds the largest index, j, for which fIDs[j] is in the range (current node's ID, joining node's ID], in modulo arithmetic, and forwards the join packet to fingers[j]. (If you find that last sentence hard to parse, as I do, work through the following example first and then re-read that last sentence and go "aha!".)

Let's look at an example (you may want to make this one of your first test cases). Say your node ID is 23 and thus your fIDs[] contains: 24, 25, 27, 31, 39, 55, 87, 151. The finger table does NOT necessarily record consecutive nodes on the identifier ring. In this example, we may have nodes with IDs 40, 43, 56. The finger table entry corresponding to fID 39 is node 40 and the entry for fID 55 is 56. If now a join request arrives with ID 42, you want to forward it to node 40 (fID 39), not node 56 (fID 55). If you send the request to node 56, you would have missed node 43, which in this case is the correct node to forward the join request to.

In the O(N) case (Lab 4), each node points to its immediate successor, with no potential for skipping unseen node(s) between one node and its successor, so we can simply forward the join request to the next node with an ID that is subsequent to that of the joining node. In the present case that uses the finger table, there is a possibility of unseen nodes and we must therefore forward the join request to the largest ID that still precedes that of the joining node in the finger table.

Finally, define two functions, I call them dhtn::fixup(int idx) and dhtn::fixdn(int idx). Given index idx, fixup() walks "up" the finger table from fingers[idx+1] to fingers[DHTN_FINGERS-1]. For each entry, k, idx < k < DHTN_FINGERS, it checks whether fIDs[k] is within the range between the node's ID and the ID of fingers[idx]. If so, it updates fingers[k] to fingers[idx]. It stops walking "up" the finger table as soon as the identifier range check above fails. Conversely, fixdn() walks "down" the finger table starting from fingers[idx-1] to fingers[0]. For each entry, k, idx > k >= 0, it checks whether the ID of fingers[idx] is equal to fIDs[k] or within the range between the fIDs[k] and the ID of fingers[k]. If so, it updates the entry of fingers[k] to be that of fingers[idx]. Unlike the fixup() case, fixdn() doesn't stop walking "down" the finger table until it reaches entry 0 or when fIDs[k] == fingers[k].dhtn_ID, to ensure that fingers in the middle of the finger table will be updated with newly learned predecessor. We stop walking down the finger table when fIDs[k] == fingers[k].dhtn_ID because in this case the entry is already correct but ID_inrange() always returns true when given the same ID as the beginning and end of range.

As in PA1, dhtdb listens to two sockets: the image socket to handle the image query from client, and the DHT socket to communicate with other dhtdb. As in Lab 4, the ID range of a node changes everytime it gets a new predecessor. The support code reloads a node's image database and resets its cache and Bloom Filter everytime the node's predecessor changes, by calling imgdb::reloaddb(begin, end).

When a dhtdb receives an image query from a netimg client, it first searches its local database (and cache) for the image. If the requested image is not found, in which case, imgdb::handleqry(), from Lab4 returns 1, it creates a DHTM_SRCH packet with itself as the originator of the query stored in the query message (dhts_node), then it computes the ID of the requested image name and stores it in the dhts_imgID field of the query packet, and copies the requested image name into the dhts_imgname field of the query packet. It then calls dhtn::forward() to forward the packet along onto the DHT. The packet format used for image search on the DHT is defined in the dhtn.h file that was part of Lab 4 support code.

Forwarding a DHTM_SRCH message follows exactly the same logic as forwarding a DHTM_JOIN message. The dhtn::forward() function in Lab 4 support code allows it to be used to forward both types of packet. When forwarding a join packet, you use the ID of the joining node to determine where to forward the packet. When forwarding a search message, you use the ID of the queried image to determine where to forward the packet. When constructing the search packet, don't forget to set the version and type fields appropriately.

Since the join process is assumed to never fail, if a join packet's ttl expired, you are prompted to re-start your test with a larger ttl value. A search packet's ttl expiring, on the other hand, is not a failure condition. Instead, your dhtdb should time out waiting for the search reply and inform the netimg client that the queried image is not found, by returning a NETIMG_NFOUND message as in PA1. Also as in PA1, when an image query cannot be satisfied locally either from the database or cache, we allow only one outstanding search to originate from each dhtdb node at any one time.

This task should take about 10 lines of new or modified code. If you've decided not to use the support code, your search message MUST use the dhtsrch_t packet format defined in dhtn.h and your code must interoperate with the provided reference implementation, refdhtdb.

Next you need to update the dhtn::handlepkt() function to recognize DHTM_SRCH packet. Upon receiving a DHTM_SRCH packet, you first check your local cache/database for the image. If the image is found, you send a DHTM_RPLY back to the query originator and you're done. If the image is not in cache, you check whether the image's ID is within your range. If so, the image doesn't exist, and you send back a DHTM_MISS to the query originator and you're done. If the image is not in cache and its ID is not in your range and the node forwarding the search to you is not expecting the ID to be within your range, you simply forward the search using dhtn::forward(). However, if the queried image's ID is not within your identifier range, but the node forwarding the search message expected it to be within your range, you must send back a DHTM_RDRT message, same as the case for handling join packets. To iterate: if a queried image is found in cache, or it is in your range but there's no such image, you simply send a DHTM_RPLY or DHTM_MISS back to the query originator. You don't forward the search message further and you don't need to fix any existing inconsistencies in your finger table (if you do, you could end up sending multiple replies to the query originator). This task should take no more than 40 lines of code.

As an originator of a search sent to the DHT, if the image is found, you receive back a DHTM_RPLY packet from another DHT node, otherwise you receive back a DHTM_MISS packet. If you receive a DHTM_MISS packet, you inform the client that image is not found by calling imgdb::handlemiss(). Unlike in PA1, a DHTM_RPLY packet doesn't actually transfer any image between DHT nodes, instead you should think of it simply as a "permission" for a node to load an image from the images folder to its database and to subsequently serve that image as if it were part of its database. When you receive a DHTM_RPLY packet, you can call imgdb::handlerply() to load the image into your database/cache and to send the image to the client. This task should take on the order of 10 lines of new code.

Testing Your Code

Support Code

So that you don't feel like you're just filling in functions and not having any chance to write your own program from scratch, we are not providing further support code beyond the above. As with PA1, if you have not been able to complete Lab 3 and would like the solution so that you can complete this assignment, you may choose to forfeit the 10 points associated with it and obtain a solution from the instructor. Similarly for Lab 4, for 30 points.

Submission Instructions

Your solution must either work with the provided Makefile or you must provide a Makefile that works on CAEN eecs489 hosts. Do NOT use any library or compiler option that is not used in the provided Makefile. Doing so would likely make your code not portable and if we can't compile your code, you will be heavily penalized. Test your compilation on CAEN eecs489 hosts! Your submission must compile and run without errors on CAEN eecs489 hosts.

Your code MUST interoperate with the provided refdhtdb.