ECS 110: Data Structures and Programming Discussion Section Notes -- Week 6 Ted Krovetz (tdkrovetz@ucdavis.edu) KRUSKAL'S ALGORITHM & RADIX SORTING =================================== This week we will make use of the Disjoint Set ADT to solve the problem of connecting vertices of a graph with minimum cost. We will also look at Radix Sorting, the type of sorting that programming assignment 3 will be based upon. 1. KRUSKAL'S ALGORITHM ---------------------- 1.1 Graphs A graph is defined by a set of vertices, V, and a set of pairs of vertices, E, called edges. Each edge connects two vertices and can have an associated cost for the connection. Graphs with these weighted edges are very useful for modelling many real life situations. As an example, consider the set of commercial airports to be your vertices, and the cost to fly from one airport to another to be your set of weighted edges. Or, to continue Prof. Rogaway's example from class, consider computers on a network to be your vertices, and the cost of directly connecting two computers with networking cable to be your weighted edges. Trees are a subset of graphs, a tree being a connected acyclic graph. We will come back to graphs later in the course. 1.2 Minimum-Cost Spanning Tree Let's suppose we have a graph with weighted edges and we want to reduce it to a connected acyclic graph with minimum total cost. Remember that connected mean that there is a path from any vertex to any other vertex, and that acyclic means that no path crosses the same vertex twice. Minimum cost means that the sum of the cost of the edges is minimal. Such a connected acyclic graph with minimum cost is known as a minimum-cost spanning tree (MST). In our computer network example, an MST would be the least expensive way to ensure that each computer could communicate with all the others. 1.3 The algorithm A simple and efficient algorithm for building a MST from a set of vertices and edges with known costs was suggested by J. B. Kruskal, and is known as Kruskal's algorithm. It is an excellent example of the Greedy strategy of algorithm design. Begin with all the vertices and no edges. Consider, one at a time, the inclusion of each edge beginning with the edge of least cost. If an edge under consideration would cause a cycle, then discard the edge. Otherwise, the edge should be included in the MST. Stop as soon as all vertices are connected. This algorithm is guaranteed to yield a MST. 1.4 Disjoint Sets The disjoint sets ADT is ideal for solving this problem. We can define set membership as meaning that vertices in a set are connected to the others in the same set. If we initially create N sets for all N vertices in the graph, with each set containing one of the N vertices, then we have successfully modelled the situation in our algorithm before any edges have been accepted to our MST. Each vertex is connected only to itself. We then consider, in ascending order, each weighted edge. If the two vertices of an edge belong to different sets, then we merge the two sets and include the edge in our MST. If the two vertices of an edge belong to the same set, then the edge's inclusion would create a cycle. Continue until N-1 edges have been accepted or all edges have been considered. (Why N-1?) In pseudo code: for each vertex v in G makeset(v) while ( ! MST & edges_remain) (u, v) = edge of minimum cost i = find(u) j = find(v) if (i != j) union(i, j); Add_To_MST(u, v); if (MST) return Success; else return Failure; 2. Radix Sorting ---------------- Programming assignment 3 will require that you understand Radix sorting. Generalized sorting algorithms like quicksort and mergesort operate in complexity O(n log n). This is as fast as these algorithms can get because they have no information about the data they are to sort except whether one item is "less" than another. 2.1 Bin Sorting However, if more information is known about the data to be sorted, then specialized sorting algorithms can do better. In particular, if all the elements are known to be in a relatively small range, a bin sort can sort the items in O(n) time. A bin sort works by allocating an array of lists. The array is as long as the known range. For example if we are to sort numbers in the range 0..99 we would allocate an array of 100 lists. Next, we read the list to be sorted. Each time a number is encountered, it is placed on the list associated with its number. Once all the numbers have been placed in the array of lists, the lists are concatenated and the numbers have been sorted. The algorithm works in O(n) time. 2.2 Radix Sorting Bin sorting can be extended to the sorting of elements with multiple keys of limited range. A perfect example of such elements are strings of characters. A string, s, which is 20 characters in length can be viewed as a sort element with 20 keys, s[0] being the most significant and s[19] being least significant. A set of strings can be bin sorted on one character, yielding a partial ordering. The resultant strings can then be sorted on a second character, yielding a partial ordering which is even closer to a total ordering. This continues until the bin sort has been performed on all character locations; in this case 20 times. Each sort being conducted in O(n) time, the total sort time is O(20 * n), which is still O(n). The radix sort can begin with the most significant key (an MSD or forward scan radix sort) or with the least significant key (LSD or backward scan). 2.2.1 MSD Radix Sort Algorithm Proceed as follows for the MSD radix sort: Split the strings into bins according to their first character. Apply the algorithm recursively on each bin separately, with the first character removed. Bins containing only one string need not be processed further. After the ith step of the algorithm, the input strings will be properly sorted on their first i characters. 2.2.2 LSD Radix Sort Algorithm Proceed as follows for the LSD radix sort: Split the strings into bins according to their last character. Apply the algorithm recursively on all strings, with the last character removed. After the ith step of the algorithm, the input strings will be properly sorted on their last i characters. Here's pseudo code for the LSD radix sort (from Aho, Hopcroft, and Ullman; 1983): procedure radixsort; /* Sorts list A of strings, each k chars long. Uses 256 bins, Bin[256] */ for (i = k-1; i >= 0; i--) { // Once for each character in s for (j = 0; j < 256; j++) empty Bin[j]; // Start with empty bins each iteration for each string s on list A move s to end of Bin[s[i]] for (j = 0; j < 256; j++) concatenate Bin[j] onto the end of A; } 3. Additional Information ------------------------- Read section 6.3.1 for more information about Kruskal's algorithm, and read section 7.7 for more about radix sorting.