?_þÉLÀ²tM°§l!onÔ8?Produced by WebTwin (www.webtwin.com) HTML->WinHelp converter.~Help file produced by WebTwin (www.webtwin.com) HTML->WinHelp converter. This text does not appear in the registered version.-CB(`topFile',"Start",`JI(`',`documentID0')')%RR("user.exe","SetWindowLong","Uii")Z mainThe Algorithm Design Manual íœñÿÿÿÿÿâ  /ÿÿÿÿ;)z4ÿÿddÿÿÿÿ|CONTEXT“""|CTXOMAP*–!|FONT…”!|KWBTREE5¢!|KWDATA5–!|KWMAP¢!|SYSTEM|TOPICï|TTLBTREEdÚ!|bm0Â2#|bm1Œ8#|bm10±Ÿ#|bm11C¥#|bm12«#|bm13 ²#|bm14ž#|bm15£Ÿ#|bm16UÆ#|bm175Ï#|bm18vÓ#|bm19Ø#|bm2N>#|bm20EÞ#|bm21‹ã#|bm22è#|bm23'ü#|bm24d$|bm25 $|bm26_ $|bm27$|bm28Õ$|bm29$$|bm3›D#|bm30Á$|bm314#$|bm32l)$|bm33/$|bm34 5$|bm35£9$|bm368>$|bm37ŠN$|bm38aS$|bm39òW$|bm4+K#|bm40„\$|bm41/a$|bm42sf$|bm43k$|bm44žy$|bm45R~$|bm46[„$|bm47‰$|bm48ԍ$|bm49ݒ$|bm5Q#|bm50:£$|bm516¬$|bm52ê°$|bm53¢µ$|bm54§»$|bm55:Á$|bm56cÆ$|bm57AË$|bm58áÐ$|bm59Sã$|bm6®W#|bm60Ùè$|bm61žõ$|bm62«ú$|bm63éÿ$|bm64%|bm65Œ %|bm66W%|bm67Š%|bm68ú%|bm69y!%|bm7`_#|bm70õ%%|bm71C,%|bm72“2%|bm73 9%|bm74>%|bm75zB%|bm76G%|bm77p^%|bm78Üu%|bm79°‰%|bm8@h#|bm80E%|bm81ȓ%|bm82™%|bm83lŸ%|bm84ª¥%|bm85qª%|bm86ð®%|bm87׳%|bm88¹%|bm89¢Ÿ%|bm9·™#|bm90µÅ%–Ž!Ž!ÿÿÿÿ LÿÿÿÿX1ÿÿÿÿÿÿÿÿXšlThe Algorithm Design ManualB š# €>€€‚ÿThe Algorithm Design Manual 7WXÑà Ÿ€€ç_xÊ«€†"€‰€æ}{`/€†"€‰€†"€çaxÊ«€†"€‰€çbxÊ«€†"€‰€æt{`/€†"€‰€æu{`/€†"€‰€æv{`/€†"€‰€€ç_xÊ«€‰€€æw{`/€‰€‚ÿ Next: Preface Up: Main PageQؚ"y#À€³ÿ?ÿ?(€€€†"€‚ÿ€€‚ÿ,€ €€‚€‚ÿ€š€‚ÿÿÿThe Algorithm Design ManualSteven S. SkienaDepartment of Computer ScienceState University of New YorkStony Brook, NY 11794-4400algorith@cs.sunysb.eduCopyright © 1997 by Springer-Verlag, New York&ÑH# €€€‚ÿ)"q& €€€‚ÿ&H—# €€€‚ÿd$qû@ P€H€ȀçÔﷀ‰€‚€çXýÉ3€‰€‚ÿu Contents u Techniques•—’ ò€€‘€€繋В€‰€‚€ç×ñ€‰€‚€ç²ͧ€‰€‚€çI’P€‰€‚€窏˜¯€‰€‚€ç Ÿ€‰€‚€çl¬¥m€‰€‚ÿu Introduction to Algorithms u Data Structures and Sorting u Breaking Problems Down u Graph Algorithms u Combinatorial Search and Heuristic Methods u Intractable Problems and Approximations u How to Design AlgorithmsBûÒ0 0€$€ȀçÍ:¬Ì€‰€‚ÿu Resources‰H[A R€€‘€€ç.ɲ+€‰€‚€çW¹Š€‰€‚ÿu A Catalog of Algorithmic Problems u Algorithmic Resources‘AÒìP p€‚€Ȁçý7ۀ‰€‚€ç5R¶€‰€‚€çQtÆH€‰€‚ÿu References u Index u About this document ...&[# €€€‚ÿ)ì;& €€€‚ÿS+Ž( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997);·& €€€‚ÿµfŽlO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1·1UÿÿÿÿÿÿÿÿÿÿÿÿÁ$lÁ" €€€ ÿ8ù1Ÿ"ÿÿÿÿÿÿÿÿù' ­‹Preface. Á' # €€€‚ÿPreface odù–  äÙ€€ç*8ۀ†"€ ‰€çhxÊ«€†"€ ‰€çhxÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æx{`/€†"€‰€æy{`/€†"€‰€æz{`/€†"€‰€€ç*8ۀ‰€€çhxÊ«€‰€€çhxÊ«€‰€‚€‚€€€‚ÿ Next: Acknowledgments Up: The Algorithm Design Manual Previous: The Algorithm Design ManualPrefaceMost of the professional programmers that I've encountered are not well prepared to tackle algorithm design problems. This is a pity, because the techniques of algorithm design form one of the core practical technologies of computer science. Designing correct, efficient, and implementable algorithms for real-world problems is a tricky business, because the successful algorithm designer needs access to two distinct bodies of knowledge:¬m' N@? L€Û€Ȁ€€€€€‚€€€‚ÿu Techniques - Good algorithm designers understand several fundamental algorithm design techniques, including data structures, dynamic programming, depth-first search, backtracking, and heuristics. Perhaps the single most important design technique is modeling, the art of abstracting a messy real-world application into a clean problem suitable for algorithmic attack.u Resources - Good algorithm designers stand on the shoulders of giants. Rather than laboring from scratch to produce a new algorithm for every task, they know how to find out what is known about a particular problem. Rather than reimplementing popular algorithms from scratch, they know where to seek existing implementations to serve as a starting point. They are familiar with a large set of basic algorithmic problems, which pr– N@Áovides sufficient source material to model most any application.!ü– oA% €ù€€‚ÿThis book is intended as a manual on algorithm design, providing access to both aspects of combinatorial algorithms technology for computer professionals and students. Thus this book looks considerably different from other books on algorithms. Why?a.N@ÐD3 4€]€Ȁ€€‚€€‚ÿu We reduce the design process to a sequence of questions to ask about the problem at hand. This provides a concrete path to take the nonexpert from an initial problem statement to a reasonable solution.u Since the practical person is usually looking for a program more than an algorithm, we provide pointers to solid implementations whenever they are available. We have collected these implementations on the enclosed CD-ROM and at one central FTP/WWW site for easy retrieval. Further, we provide recommendations to make it easier to identify the correct code for the job. With these implementations available, the critical issue in algorithm design becomes properly modeling your application, more so than becoming intimate with the details of the actual algorithm. This focus permeates the entire book.ïoAçF( €ß€Ȁ€‚ÿu Since finding out what is known about a problem can be a difficult task, we provide a catalog of important algorithmic problems as a major component of this book. By browsing through this catalog, the reader can quickly identify what their problem is called, what is known about it, and how they should proceed to solve it. To aid in problem identification, we include a pair of ``before'' and ``after'' pictures for each problem, illustrating the required input and output specifications.8 ÐDJ/ ,€€Ȁ€‚€€‚ÿu For each problem in the catalog, we provide an honest and convincing motivation, showing how it arises in practice. If we could not find such an application, then the problem doesn't appear in this book.u In practice, algorithm problems do not arise at the beginning of a large project. Rather, they typically arise as subproblems when it suddenly becomes clear that the programmer does not know how to proceed or that the current program is inadequate. To provide a better perspective on how algorithm problems arise in the real world, we include a collection of ``war stories,'' tales from our experience on real problems. The moral of these stories is that algorithm design and analysis is not just theory, but an important tool to be pulled out and used as needed.N)çFmL% €S€€‚ÿEqually important is what we do not do in this book. We do not stress the mathematical analysis of algorithms, leaving most of the analysis as informal arguments. You will not find a single theorem anywhere in this book. Further, we do not try to be encyclopedic in our descriptions of algorithms, but only in our pointers to descriptions of algorithms. When more details are needed, the reader should follow the given references or study the cited programs. The goal of this manual is to get you going in the right direction as quickly as possible.úJŒM% €õ€€‚ÿBut what is a manual without software? This book comes with a substantial electronic supplement, an ISO-9660 compatible, multiplatform CD-ROM, which can be viewed using Netscape, Microsoft Explorer, or any other WWW browser. This CD-ROM contains:v6mL@ N€m€Ȁ€€‚€€‚€€€€‚ÿu A complete hypertext version of the full printed book. Indeed, the extensive cross-references within the book are best followed using the hypertext version.u The source code and URLs for all cited implementations, mirroring the Stony Brook Algorithm Repository WWW site. Programs in C, C++, Fortran, and Pascal are included, providing an average of four different implementations for each algorithmic problem.u More than ten hours of audio lectures on the design and analysis of algorithms are provided, all keyed to the on-line lecture notes. FollowŒMÁing these lectures provides another approach to learning algorithm design techniques. These notes are linked to an additional twenty hours of audio over the WWW. Listening to all the audio is analogous to taking a one-semester college course on algorithms!ÑŒŒM߃E X€€€€€€€‚€€€€€€‚ÿThis book is divided into two parts, techniques and resources. The former is a general guide to techniques for the design and analysis of computer algorithms. The resources section is intended for browsing and reference, and comprises the catalog of algorithmic resources, implementations, and an extensive bibliography.Altogether, this book covers material sufficient for a standard Introduction to Algorithms course, albeit one stressing design over analysis. We assume the reader has completed the equivalent of a second programming course, typically titled Data Structures or Computer Science II. Textbook-oriented features include:7ý‡: B€û€Ȁ€€‚€€‚€€‚ÿu In addition to standard pen-and-paper exercises, this book includes ``implementation challenges'' suitable for teams or individual students. These projects and the applied focus of the text can be used to provide a new laboratory focus to the traditional algorithms course. More difficult exercises are marked by (*) or (**).u ``Take-home lessons'' at the beginning of each chapter emphasize the concepts to be gained from the chapter.u This book stresses design over analysis. It is suitable for both traditional lecture courses and the new ``active learning'' method, where the professor does not lecture but instead guides student groups to solve real problems. The ``war stories'' provide an appropriate introduction to the active learning method.ši߃°ˆ1 0€Õ€Ȁ€†"€‚ÿu A full set of lecture slides for teaching this course is available on the CD-ROM and via the World Wide Web, both keyed to unique on-line audio lectures covering a full-semester algorithm course. Further, a complete set of my videotaped lectures using these slides is available for interested parties. See http://www.cs.sunysb.edu/ algorith for details.,‡Üˆ' € €€€‚ÿ)°ˆ‰& €€€‚ÿÊÉ܈ϊ Ё£€€‚ç*8ۀ†"€ ‰€çhxÊ«€†"€ ‰€çhxÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ{{`/€†"€‰€æ|{`/€†"€‰€憌4õ€†"€‰€€ç*8ۀ‰€€çhxÊ«€‰€€çhxÊ«€‰€€‚‚ÿ Next: Acknowledgments Up: The Algorithm Design Manual Previous: The Algorithm Design Manual AlgorithmsMon Jun 2 23:33:50 EDT 1997)‰øŠ& €€€‚ÿµfϊ­‹O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1øŠÞ‹1UÿÿÿÿÿÿÿÿÿÿÿÿދŒ$­‹Œ" €€€ ÿ9ދ;Œ1Ù5ÿÿÿÿÿÿÿÿ;ŒjŒ@Contents/ ŒjŒ# €€€‚ÿContents uŠ;Œßë €#€€çexÊ«€†"€ ‰€çhxÊ«€†"€ ‰€ç!8ۀ†"€ ‰€çbxÊ«€†"€ ‰€æ}Œ4õ€†"€‰€æ~Œ4õ€†"€‰€æŒ4õ€†"€‰€€çexÊ«€‰€€çhxÊ«€‰€€ç!8ۀ‰€‚€ ‚ÿ Next: Techniques Up: The Algorithm Design Manual Previous: CaveatContentsIjŒ(Ž4 8€*€Ȁ€çXýÉ3€‰€‚ÿu TechniquesT#ߍ|Ž1 2€F€‘€€繋В€‰€‚ÿu Introduction to AlgorithmsT#(ŽÐŽ1 2€F€ـ€çõy²¥€‰€‚ÿu Correctness and Efficiencyh'|Ž8A R€N€!€çV¹€‰€‚€緖¿c€‰€‚ÿu Correctness u Efficiencyu4Ў­A R€h€ـ€ç%Æ€‰€‚€çy³Ì!€‰€‚ÿu Expressing Algorithms u Keeping Score—V8PÀA R€¬€!€çÚAӀ€‰€‚€ç;ÐÙ߀‰€‚ÿu The RAM Model­PÀŒ of Computation u Best, Worst, and Average-Case ComplexityGµ­—Á’ ò€k€ـ€çœ^à>€‰€‚€çHה€‰€‚€çßÕÅ<€‰€‚€ç@d̛€‰€‚€ç¡òÒú€‰€‚€çÙY€‰€‚€çcàž€‰€‚ÿu The Big Oh Notation u Growth Rates u Logarithms u Modeling the Problem u About the War Stories u War Story: Psychic Modeling u ExercisesU$PÀìÁ1 2€H€‘€€ç×ñ€‰€‚ÿu Data Structures and SortingP—Á<Â1 2€>€ـ€çĝ怉€‚ÿu Fundamental Data Typesœ\ìÁùÂa ’€ž€!€ç%,ív€‰€‚€熺óՀ‰€‚€ççHú4€‰€‚€ç“ÁŠ€‰€‚ÿu Containers u Dictionaries u Binary Search Trees u Priority QueuesÓr<ÂÌÃa ’€ä€ـ€ç*Àß2€‰€‚€ç‹N摀‰€‚€çìÜìð€‰€‚€çMkóO€‰€‚ÿu Specialized Data Structures u Sorting u Applications of Sorting u Approaches to Sortingö„ùÂÂÄr ²€ €!€ç®ùù®€‰€‚€çˆ€‰€‚€çpm€‰€‚€çÑ€ ̀‰€‚€ç23+€‰€‚ÿu Data Structures u Incremental Insertion u Divide and Conquer u Randomization u Bucketing TechniquesìŠÌîÅb ’€€ـ€çÞ«4€€‰€‚€çuªù(€‰€‚€çÖ8ˆ€‰€‚€ç7Ç瀉€‚ÿu War Story: Stripping Triangulations u War Story: Mystery of the Pyramids u War Story: String 'em Up u ExercisesPÂÄþÅ1 2€>€‘€€ç²ͧ€‰€‚ÿu Breaking Problems DownM®ÅKÆ1 2€8€ـ€ç˜U F€‰€‚ÿu Dynamic Programming¥þÅbÇr ²€K€!€çù㥀‰€‚€çZr€‰€‚€ç»!c€‰€‚€ç'€‰€‚€ç}.!€‰€‚ÿu Fibonacci numbers u The Partition Problem u Approximate String Matching u Longest Increasing Sequence u Minimum Weight Triangulation?ÍKÆ¡Èr ²€›€ـ€ç)–Nv€‰€‚€çÀ”€‰€‚€ç!#~€‰€‚€炱 ݀‰€‚€çã?'<€‰€‚ÿu Limitations of Dynamic Programming u War Story: Evolution of the Lobster u War Story: What's Past is Prolog u War Story: Text Compression for Bar Codes u Divide and Conquer¢QbÇCÉQ r€¢€!€çDÎ-›€‰€‚€ç¥\4ú€‰€‚€çë:Y€‰€‚ÿu Fast Exponentiation u Binary Search u Square and Other RootsC¡È†É1 2€$€ـ€çgyAž€‰€‚ÿu ExercisesJCÉÐÉ1 2€2€‘€€çI’P€‰€‚ÿu Graph Algorithms ɲÊb ’€€ـ€çÈH€‰€‚€çt€hl€‰€‚€ç -€‰€‚€çl 4t€‰€‚ÿu The Friendship Graph u Data Structures for Graphs u War Story: Getting the Graph u Traversing a Graphy8ÐÉ+ËA R€p€!€ç͛:Ӏ‰€‚€ç.*A2€‰€‚ÿu Breadth-First Search u Depth-First SearchY(²Ê„Ë1 2€P€ـ€珞G‘€‰€‚ÿu Applications of Graph Traversal”+ˊÌr ²€)€!€çðFNð€‰€‚€çQÕTO€‰€‚€ç²c[®€‰€‚€çòa €‰€‚€ç¿j‚b€‰€‚ÿu Connected Components u Tree and Cycle Detection u Two-Coloring Graphs u Topological Sorting u Articulation Vertices€?„Ë ÍA R€~€ـ€çViG €‰€‚€ç·÷Mj€‰€‚ÿu Modeling Graph Problems u Minimum Spanning Treesv5ŠÌ€ÍA R€j€!€ç†Tɀ‰€‚€çy[(€‰€‚ÿu Prim's Algorithm u Kruskal's AlgorithmH ÍÈÍ1 2€.€ـ€çÚ¢a‡€‰€‚ÿu Shortest Paths~=€ÍFÎA R€z€!€ç;1h怉€‚€眿nE€‰€‚ÿu Dijkstra's Algorithm u All-Pairs Shortest Path°_ÈÍöÎQ r€Ÿ€ـ€çýMu€€‰€‚€ç^Ü{€‰€‚€ç UœX€‰€‚ÿu War Story: Nothing but Nets u War Story: Dialing for Documents u Exercisesd3FÎZÏ1 2€f€‘€€窏˜¯€‰€‚ÿu Combinatorial Search and Heuristic MethodsFöΠÏ1 2€*€ـ€ç¡Sa€‰€‚ÿu BacktrackingÂqZÏnQ r€â€!€çâg`€‰€‚€çcpn¿€‰€‚€çÄþt€‰€‚ÿu Constructi ÏnŒng All Subsets u Constructing All Permutations u Constructing All Paths in a GraphÙx ÏGa ’€ð€ـ€ç%{}€‰€‚€ç†‚Ü€‰€‚€ç穈;€‰€‚€çH8š€‰€‚ÿu Search Pruning u Bandwidth Minimization u War Story: Covering Chessboards u Heuristic MethodsMn”1 2€8€!€ç©Æ•ù€‰€‚ÿu Simulated AnnealingÐoGda ’€Þ€i€ç’fê耉€‚€ç)e¯‘€‰€‚€çŠóµð€‰€‚€ç끌O€‰€‚ÿu Traveling Salesman Problem u Maximum Cut u Independent Set u Circuit Board Placementt3”ØA R€f€!€çL€‚€ç­žÉ €‰€‚ÿu Neural Networks u Genetic AlgorithmsÙxd±a ’€ð€ـ€ç-Ðl€‰€‚€ço»Öˀ‰€‚€çÐIÝ*€‰€‚€ç1Ø㉀‰€‚ÿu War Story: Annealing Arrays u Parallel Algorithms u War Story: Going Nowhere Fast u Exercisesa0Ø1 2€`€‘€€ç Ÿ€‰€‚ÿu Intractable Problems and Approximations{:±A R€t€ـ€çï)B€‰€‚€ç†(Æꀉ€‚ÿu Problems and Reductions u Simple ReductionsžgEQ r€Î€!€çç¶ÌI€‰€‚€çHEÓš€‰€‚€ç©ÓÙ€‰€‚ÿu Hamiltonian Cycles u Independent Set and Vertex Cover u Clique and Independent SetH1 2€.€ـ€ç bàf€‰€‚ÿu Satisfiability€?E A R€~€!€çkðæŀ‰€‚€çÌ~í$€‰€‚ÿu The Theory of NP-Completeness u 3-SatisfiabilityN[1 2€:€ـ€ç- ôƒ€‰€‚ÿu Difficult Reductionsr1 ÍA R€b€!€玛ú €‚€ç:8€‰€‚ÿu Integer Programming u Vertex Coverô’[Áb ’€%€ـ€çÑàà€‰€‚€ç2¡æ?€‰€‚€ç“/힀‰€‚€çôœóý€‰€‚ÿu Other NP-Complete Problems u The Art of Proving Hardness u War Story: Hard Against the Clock u Approximation AlgorithmsLÍNA R€˜€!€çULú\€‰€‚€ç¶ÚŒ€‰€‚ÿu Approximating Vertex Cover u The Euclidean Traveling SalesmanCÁ‘1 2€$€ـ€çi€‰€‚ÿu ExercisesR!Nã1 2€B€‘€€çl¬¥m€‰€‚ÿu How to Design AlgorithmsB‘% 0 0€$€ȀçÍ:¬Ì€‰€‚ÿu Resources[*〠1 2€T€‘€€ç.ɲ+€‰€‚ÿu A Catalog of Algorithmic ProblemsI% É 1 2€0€ـ€çx÷ z€‰€‚ÿu Data Structures˜€ ã ‚ Ҁ1€!€çمـ‰€‚€ç…þ4.€‰€‚€çýùր‰€‚€ç}‹6€‰€‚€çÞ•€‰€‚€ç?š ô€‰€‚ÿu Dictionaries u Priority Queues u Suffix Trees and Arrays u Graph Data Structures u Set Data Structures u Kd-TreesLÉ / 1 2€6€ـ€ç 6S€‰€‚ÿu Numerical ProblemsWã H  R¯€!€çŲ€‰€‚€çbS!€‰€‚€çÃá'p€‰€‚€ç$p.π‰€‚€çÐèN$€‰€‚€çgç̀‰€‚€çÈu,€‰€‚€ç)!‹€‰€‚€犒'ꀉ€‚€çë .I€‰€‚ÿu Solving Linear Equations u Bandwidth Reduction u Matrix Multiplication u Determinants and Permanents u Constrained and Unconstrained Optimization u Linear Programming u Random Number Generation u Factoring and Primality Testing u Arbitrary-Precision Arithmetic u Knapsack Problem T#/ œ 1 2€F€!€çL¯4š€‰€‚ÿu Discrete Fourier TransformPH ì 1 2€>€ـ€ç­=;€‰€‚ÿu Combinatorial ProblemsÃœ ¯ R€!€çÌAf€‰€‚€çoZHŀ‰€‚€çÓh€‰€‚€ç²Ñ-À‰€‚€ç`4"€‰€‚€çtî:€‰€‚€çÕ|Aà€‰€‚€ç6 H?€‰€‚€痙Nž€‰€‚€çø'Uý€‰€‚ÿu Sorting u Searching u Median and Selection u Generating Permutations u Generating Subsets u Generating Partitions u Generating Graphs u Calendrical Calculations u Job Scheduling u SatisfiabilityY(ì @1 2€P€ـ€çY¶[\€‰€‚ÿu Graph Problems: Polynomial-¯@ŒTime?¯%BÒ r€!€çºDb»€‰€‚€çfœ‚€‰€‚€çý»G¹€‰€‚€ç^JN€‰€‚€ç¿ØTw€‰€‚€ç g[ր‰€‚€çõa5€‰€‚€çâƒh”€‰€‚€çCo󀉀‚€瀠uR€‰€‚€ç/|±€‰€‚ÿu Connected Components u Topological Sorting u Minimum Spanning Tree u Shortest Path u Transitive Closure and Reduction u Matching u Eulerian Cycle / Chinese Postman u Edge and Vertex Connectivity u Network Flow u Drawing Graphs Nicely u Drawing Trees [*@€B1 2€T€!€籧œ€‰€‚ÿu Planarity Detection and EmbeddingW&%B×B1 2€L€ـ€çHŠa¯€‰€‚ÿu Graph Problems: Hard Problemsá€BžDÒ r€!€ç©4h€‰€‚€ç Ãnm€‰€‚€çkQù‰€‚€çÌß{+€‰€‚€ç-n‚Š€‰€‚€çŽüˆé€‰€‚€çH€‰€‚€çP–§€‰€‚€çü‘¶ü€‰€‚€瓐{¥€‰€‚€çô‚€‰€‚ÿu Clique u Independent Set u Vertex Cover u Traveling Salesman Problem u Hamiltonian Cycle u Graph Partition u Vertex Coloring u Edge Coloring u Graph Isomorphism u Steiner Tree u Feedback Edge/Vertex SetP×BE1 2€>€ـ€çU­ˆc€‰€‚ÿu Computational Geometry&DžD.G⠒‰€!€ç¶;Â€‰€‚€çʕ!€‰€‚€çxXœ€€‰€‚€çÙæ¢ß€‰€‚€ç:u©>€‰€‚€ç›°€‰€‚€çG|Ðò€‰€‚€çÞz•›€‰€‚€ç? œú€‰€‚€砗¢Y€‰€‚€ç&©ž€‰€‚€çbŽ¯€‰€‚ÿu Robust Geometric Primitives u Convex Hull u Triangulation u Voronoi Diagrams u Nearest Neighbor Search u Range Search u Point Location u Intersection Detection u Bin Packing u Medial-Axis Transformation u Polygon Partitioning u Simplifying Polygons ÎmEüGa ’€Ú€!€çÃB¶v€‰€‚€ç$ьՀ‰€‚€ç…_Ã4€‰€‚€çæíɓ€‰€‚ÿu Shape Similarity u Motion Planning u Maintaining Line Arrangements u Minkowski SumQ .GMH1 2€@€ـ€ç+ÁD?€‰€‚ÿu Set and String Problems±ÿüGþI² 2ÿ€!€ç¿ 耉€‚€ç#NG€‰€‚€ç„ÜŠ€‰€‚€çåj€‰€‚€çFù#d€‰€‚€秇*À‰€‚€ç1"€‰€‚€çi€7€‰€‚€çÊ2>à€‰€‚ÿu Set Cover u Set Packing u String Matching u Approximate String Matching u Text Compression u Cryptography u Finite State Machine Minimization u Longest Common Substring u Shortest Common SuperstringOMHMJ1 2€<€‘€€çW¹Š€‰€‚ÿu Algorithmic ResourcesJþI—J1 2€2€ـ€爄[˜€‰€‚ÿu Software systems]MJôJA R€8€!€çƒ A€‰€‚€ç€' €‰€‚ÿu LEDA u NetlibY(—JMK1 2€P€i€çáŸ-ÿ€‰€‚ÿu Collected Algorithms of the ACMÛzôJ(La ’€ô€!€çB.4^€‰€‚€磌:œ€‰€‚€çKA€‰€‚€çeÙG{€‰€‚ÿu The Stanford GraphBase u Combinatorica u Algorithm Animations with XTango u Programs from Books…MK­M‚ Ҁ€i€çÆgNڀ‰€‚€ç'öT9€‰€‚€çÓnuŽ€‰€‚€çjm:7€‰€‚€çËû@–€‰€‚€ç,ŠGõ€‰€‚ÿu Discrete Optimization Algorithms in Pascal u Handbook of Data Structures and Algorithms u Combinatorial Algorithms for Computers and Calculators u Algorithms from P to NP u Computational Geometry in C u Algorithms in C++’A(L?NQ r€‚€ـ€çNT€‰€‚€çîŠT³€‰€‚€çO5[€‰€‚ÿu Data Sources u Textbooks u On-Line Resources„3­MÃNQ r€f€!€ç°Ãaq€‰€‚€çRhЀ‰€‚€çràn/€‰€‚ÿu Literature u People u SoftwareZ)?NO1 2€R€ـ€çY„€‰€‚ÿu Professional Consulting Services‘AÃN®OP p€‚€Ȁçý7ۀ‰€‚€ç5R¶€‰€‚€çQtÆH€‰€‚ÿu References u Index u About this document ...,OÚO' € €€€‚ÿ)®O€& €€€‚ÿÚO€ŒS+ÚOb€( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)€‹€& €€€‚ÿµfb€@O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1‹€q1Uÿÿÿÿÿÿÿÿÿÿÿÿq•$@•" €€€ ÿ6qˁ1> ÿÿÿÿÿÿÿÿˁ÷ŒIndex, •÷# €€€‚ÿIndex )lˁ „œ Há€€çþ7ۀ†"€ ‰€çhxÊ«€†"€ ‰€çý7ۀ†"€ ‰€çaxÊ«€†"€ ‰€€çþ7ۀ‰€€çhxÊ«€‰€€çý7ۀ‰€‚€ ‚€çhxÊ«€‰€‚ÿ Next: About this document Up: The Algorithm Design Manual Previous: ReferencesIndexThis index provides fast access to important keywords and topics in the on-line Book. The menu below partitions the index entries by the first letter of the alphabet, for ease of access. A full document index is also provided.^÷~…@ N€=€€‚æ€Œ4õ€‰€恌4õ€‰€‚ÿBe aware that the index pointer typically resides at the end of the relevant paragraph in the document, so we recommend scrolling once towards the front of the document before reading.A similar index has been provided for the Lecture Notes, which may also be of interest.>8 „Œ†#܁p@ÿÿÿÿÿÿÿÿ(€€çãm_ɀ ‰€ ‚ÿ(€€çäm_ɀ ‰€ ‚ÿ(€€çåm_ɀ ‰€ ‚ÿ(€*€çæm_ɀ ‰€ ‚ÿ(€8€ççm_ɀ ‰€ ‚ÿ(€F€çèm_ɀ ‰€ ‚ÿ(€T€çém_ɀ ‰€ ‚ÿ(€b€çêm_ɀ ‰€ ‚ÿÿÿABCDEFGH>8~…ú‡#܁p@ÿÿÿÿÿÿÿÿ(€€çn_ɀ ‰€ ‚ÿ(€€ç n_ɀ ‰€ ‚ÿ(€€çn_ɀ ‰€ ‚ÿ(€*€çn_ɀ ‰€ ‚ÿ(€8€çn_ɀ ‰€ ‚ÿ(€F€çn_ɀ ‰€ ‚ÿ(€T€çn_ɀ ‰€ ‚ÿ(€b€çn_ɀ ‰€ ‚ÿÿÿIJKLMNOP>8Œ†8‰#܁p@ÿÿÿÿÿÿÿÿ(€€çn_ɀ ‰€ ‚ÿ(€€çn_ɀ ‰€ ‚ÿ(€€çAn_ɀ ‰€ ‚ÿ(€*€ç8n_ɀ ‰€ ‚ÿ(€8€ç9n_ɀ ‰€ ‚ÿ(€F€ç:n_ɀ ‰€ ‚ÿ(€T€ç;n_ɀ ‰€ ‚ÿ(€b€çn_ɀ ‰€‚ÿ€€‚ÿ€ €‚ÿ€$€‚ÿ€(€‚ÿ€,€‚ÿ€0€‚ÿÿÿYZšb8‰ªŠ8 @€Ä€€‚ç?n_ɀ‰€€€‚ÿComplete Index (note: the complete index is large; it will take a bit of time to load))Šӊ& €€€‚ÿS+ªŠ&‹( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)ӊO‹& €€€‚ÿµf&‹ŒO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1O‹5Œ1Uÿÿÿÿÿÿÿÿÿÿÿÿ5ŒYŒ$ŒYŒ" €€€ ÿ; 5Œ”Œ1¬ÿÿÿÿÿÿÿÿ”ŒŌ—ÍTechniques1YŒŌ# €€€‚ÿTechniques °š”ŒuŽ ށa€€çfxÊ«€†"€ ‰€çhxÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ‚Œ4õ€†"€‰€惌4õ€†"€‰€æ„Œ4õ€†"€‰€€çfxÊ«€‰€€çhxÊ«€‰€€çaxÊ«€‰€‚€‚€€‚ÿ Next: Introduction to Algorithms Up: The Algorithm Design Manual Previous: ContentsTechniques)ŌžŽ& €€€‚ÿ&uŽĎ# €€€‚ÿS#žŽ0 0€F€Ȁ繋В€‰€‚ÿu Introduction to AlgorithmsÎ ĎñÀ R€‘€€çõy²¥€‰€‚€ç%Æ€‰€‚€çy³Ì!€‰€‚€çœ^à>€‰€‚€çHה€‰€‚€çßÕÅ<€‰€‚€ç@d̛€‰€‚€ç¡òÒú€‰€‚€çÙY€‰€‚€çcàž€‰€‚ÿu Correctness and Efficiency u ñÀYŒExpressing Algorithms u Keeping Score u The Big Oh Notation u Growth Rates u Logarithms u Modeling the Problem u About the War Stories u War Story: Psychic Modeling u ExercisesT$EÁ0 0€H€Ȁç×ñ€‰€‚ÿu Data Structures and SortingÍñÀò 27€‘€€çĝ怉€‚€ç*Àß2€‰€‚€ç‹N摀‰€‚€çìÜìð€‰€‚€çMkóO€‰€‚€çÞ«4€€‰€‚€çuªù(€‰€‚€çÖ8ˆ€‰€‚€ç7Ç瀉€‚ÿu Fundamental Data Types u Specialized Data Structures u Sorting u Applications of Sorting u Approaches to Sorting u War Story: Stripping Triangulations u War Story: Mystery of the Pyramids u War Story: String 'em Up u ExercisesOEÁaÃ0 0€>€Ȁç²ͧ€‰€‚ÿu Breaking Problems DownûÃîÄ’ ò€÷€‘€€ç˜U F€‰€‚€ç)–Nv€‰€‚€çÀ”€‰€‚€ç!#~€‰€‚€炱 ݀‰€‚€çã?'<€‰€‚€çgyAž€‰€‚ÿu Dynamic Programming u Limitations of Dynamic Programming u War Story: Evolution of the Lobster u War Story: What's Past is Prolog u War Story: Text Compression for Bar Codes u Divide and Conquer u ExercisesIaÃ7Å0 0€2€ȀçI’P€‰€‚ÿu Graph AlgorithmsLîÄEÇ R™€‘€€çÈH€‰€‚€çt€hl€‰€‚€ç -€‰€‚€çl 4t€‰€‚€珞G‘€‰€‚€çViG €‰€‚€ç·÷Mj€‰€‚€çÚ¢a‡€‰€‚€çýMu€€‰€‚€ç^Ü{€‰€‚ÿu The Friendship Graph u Data Structures for Graphs u War Story: Getting the Graph u Traversing a Graph u Applications of Graph Traversal u Modeling Graph Problems u Minimum Spanning Trees u Shortest Paths u War Story: Nothing but Nets u War Story: Dialing for Documents C7ňÇ1 2€$€‘€€ç UœX€‰€‚ÿu Exercisesc3EÇëÇ0 0€f€Ȁ窏˜¯€‰€‚ÿu Combinatorial Search and Heuristic Methods·ˆÇ¢É² 2 €‘€€ç¡Sa€‰€‚€ç%{}€‰€‚€ç†‚Ü€‰€‚€ç穈;€‰€‚€çH8š€‰€‚€ç-Ðl€‰€‚€ço»Öˀ‰€‚€çÐIÝ*€‰€‚€ç1Ø㉀‰€‚ÿu Backtracking u Search Pruning u Bandwidth Minimization u War Story: Covering Chessboards u Heuristic Methods u War Story: Annealing Arrays u Parallel Algorithms u War Story: Going Nowhere Fast u Exercises`0ëÇÊ0 0€`€Ȁç Ÿ€‰€‚ÿu Intractable Problems and ApproximationsÄ¢ÉÆ˲ 2%€‘€€çï)B€‰€‚€ç†(Æꀉ€‚€ç bàf€‰€‚€ç- ôƒ€‰€‚€çÑàà€‰€‚€ç2¡æ?€‰€‚€ç“/힀‰€‚€çôœóý€‰€‚€çi€‰€‚ÿu Problems and Reductions u Simple Reductions u Satisfiability u Difficult Reductions u Other NP-Complete Problems u The Art of Proving Hardness u War Story: Hard Against the Clock u Approximation Algorithms u ExercisesQ!ÊÌ0 0€B€Ȁçl¬¥m€‰€‚ÿu How to Design Algorithms&ÆË=Ì# €€€‚ÿ)ÌfÌ& €€€‚ÿS+=̹Ì( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)fÌâÌ& €€€‚ÿµf¹Ì—ÍO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1âÌÈÍ1Uÿÿÿÿÿÿÿÿ ÿÿÿÿÈÍìÍ$—ÍìÍ" €€€ ÿKÈÍ7Î1íÿÿÿÿÿÿÿÿ 7ÎxÎÚDIntroduction to AlgorithmsAìÍxÎ# €<€€‚ÿIntroduction to Algorithms )$7έ ؁Y€€ç"8ۀ†"€ ‰€çexÊ«€†"€ ‰€çexÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ…Œ4õ€†"€‰€汌4õ€†"€‰€暌4õ€†"€‰€€ç"8ۀ‰€€çexÊ«€‰€€çexÊ«€‰€‚€‚€‚ÿ Next: Correctness and Efficiency Up: Techniques Previous: TechniquexέìÍsIntroduction to AlgorithmsWhat is an algorithm? An algorithm is a procedure to accomplish a specific task. It is the idea behind any computer program.1àxÎÞQ p€Ã€€€€€€€€€‚€€€†"€€‚ÿTo be interesting, an algorithm has to solve a general, well-specified problem. An algorithmic problem is specified by describing the complete set of instances it must work on and what properties the output must have as a result of running on one of these instances. This distinction between a problem and an instance of a problem is fundamental. For example, the algorithmic problem known as sorting is defined as follows:        Input: A sequence of n keys . ¥Z­ƒK d€·€€€†"€€€€€‚€€€€‚ÿOutput: The permutation (reordering) of the input sequence such that . An instance of sorting might be an array of names, such as {Mike, Bob, Sally, Jill, Jan}, or a list of numbers like {154, 245, 568, 324, 654, 324}. Determining whether you in fact have a general problem to deal with, as opposed to an instance of a problem, is your first step towards solving it. This is true in algorithms as it is in life.An algorithm is a procedure that takes any of the possible input instances and transforms it to the desired output. There are many different algorithms for solving the problem of sorting. For example, one method for sorting starts with a single element (thus forming a trivially sorted list) and then incrementally inserts the remaining elements so that the list stays sorted. This algorithm, insertion sort, is described below:  ‘Þ’~ ʀ#€€‚‚€€€€‚€€€€‚€€€€€€€€€€€€€€€€‚ÿ InsertionSort(A) for i = 1 to n-1 do for j = i+1 to 2 do if (A[j] < A[j-1]) then swap(A[j],A[j-1]) 䃝 ' €É€€‚‚ÿNote the generality of this algorithm. It works equally well on names as it does on numbers, given the appropriate < comparison operation to test which of the two keys should appear first in sorted order. Given our definition of the sorting problem, it can be readily verified that this algorithm correctly orders every possible input instance.In this chapter, we introduce the desirable properties that good algorithms have, as well as how to measure whether a given algorithm achieves these goals. Assessing algorithmic performance requires a modest amount of mathematical notation, which we also present. Although initially intimidating, this notation proves essential for us to compare algorithms and design more efficient ones.  ýؒš % €±€€‚ÿWhile the hopelessly ``practical'' person may blanch at the notion of theoretical analysis, we present this material because it is useful. In particular, this chapter offers the following ``take-home'' lessons:  Í · P n€€Ȁ€€‚€€‚€€‚€€†"€€€‚ÿu Reasonable-looking algorithms can easily be incorrect. Algorithm correctness is a property that must be carefully demonstrated.u Algorithms can be understood and studied in a machine independent way.u The ``big Oh'' notation and worst-case analysis are tools that greatly simplify our ability to compare the efficiency of algorithms.u We seek algorithms whose running times grow logarithmically, because grows very slowly with increasing n.°ˆš g( €€Ȁ€‚ÿu Modeling your application in terms of well-defined structures and algorithms is the most important single step towards a solution.,· “' € €€€‚ÿ)gŒ& €€€‚ÿ&“â# €€€‚ÿS#Œ50 0€F€Ȁçõy²¥€‰€‚ÿu Correctness and Efficiencyh'âA R€N€‘€€çV¹€‰€‚€緖¿c€‰€‚ÿu Correctness u Efficiencyt45@@ P€h€Ȁç%Æ€‰€‚€çy³Ì!€‰€‚ÿu Expressing Algorithms u @ìÍKeeping Score—VŽ@A R€¬€‘€€çÚAӀ€‰€‚€ç;ÐÙ߀‰€‚ÿu The RAM Model of Computation u Best, Worst, and Average-Case ComplexityFµ@úA‘ ð€k€Ȁçœ^à>€‰€‚€çHה€‰€‚€çßÕÅ<€‰€‚€ç@d̛€‰€‚€ç¡òÒú€‰€‚€çÙY€‰€‚€çcàž€‰€‚ÿu The Big Oh Notation u Growth Rates u Logarithms u Modeling the Problem u About the War Stories u War Story: Psychic Modeling u Exercises&Ž@ B# €€€‚ÿ)úAIB& €€€‚ÿ³² BüC Ёu€€‚ç"8ۀ†"€ ‰€çexÊ«€†"€ ‰€çexÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ©Œ4õ€†"€‰€檌4õ€†"€‰€æ«Œ4õ€†"€‰€€ç"8ۀ‰€€çexÊ«€‰€€çexÊ«€‰€€‚‚ÿ Next: Correctness and Efficiency Up: Techniques Previous: Techniques AlgorithmsMon Jun 2 23:33:50 EDT 1997)IB%D& €€€‚ÿµfüCÚDO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1%D E1Uÿÿÿÿÿÿÿÿ ÿÿÿÿ E/E$ÚD/E" €€€ ÿL E{E16ÿÿÿÿÿÿÿÿ {EœEŠData Structures and SortingB/EœE# €>€€‚ÿData Structures and Sorting õä{E²J ðÙ€€çQ8ۀ†"€ ‰€çexÊ«€†"€ ‰€ç@n_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€欌4õ€†"€‰€æ­Œ4õ€†"€‰€殌4õ€†"€‰€€çQ8ۀ‰€€çexÊ«€‰€€ç@n_ɀ‰€‚€‚€€€€€‚ÿ Next: Fundamental Data Types Up: Techniques Previous: Implementation ChallengesData Structures and SortingWhen things go right, changing a data structure in a slow program works the same way an organ transplant does in a sick patient. For several classes of abstract data types, such as containers, dictionaries, and priority queues, there exist many different but functionally equivalent data structures that implement the given data type. Changing the data structure does not change the correctness of the program, since we presumably replace a correct implementation with a different correct implementation. However, because the new implementation of the data type realizes different tradeoffs in the time to execute various operations, the total performance of an application can improve dramatically. Like a patient in need of a transplant, only one part might need to be replaced in order to fix the problem.    $ùœEÖM+ $€ó€€€€‚ÿIt is obviously better to be born with a good heart than have to wait for a replacement. Similarly, the maximum benefit from good data structures results from designing your program around them in the first place. Still, it is important to build your programs so that alternative implementations can be tried. This involves separating the internals of the data structure (be it a tree, a hash table, or a sorted array) from its interface (operations like search, insert, delete). Such data abstraction is an important part of producing clean, readable, and modifiable programs. We will not dwell on such software engineering issues here, but such a design is critical if you are to experiment with the impact of different implementations on performance.  Ì²J¢O< F€#€€çÌAf€†"€‰€‚‚ÿIn this chapter we will also discuss sorting, stressing how sorting can be applied to solve other problems more than the details of specific sorting algorithms. In this sense, sorting behaves more like a data structure than a problem in its own right. Sorting is also represented by a significant entry in the problem catalog; namely Section .The key take-home lessons of this chapter are:KÖMù: B€#€Ȁ€€‚€€‚€€‚ÿu Building algorithms around dat¢Où/Ea structures such as dictionaries and priority queues leads to both clean structure and good performance.u Picking the wrong data structure for the job can be disastrous in terms of performance. Picking the very best data structure is often not as critical, for there are typically several choices that perform similarly.u Sorting lies at the heart of many different algorithms. Sorting the data is one of the first things any algorithm designer should try in the quest for efficiency.óË¢Oì‚( €—€Ȁ€‚ÿu Sorting can be used to illustrate most algorithm design paradigms. Data structure techniques, divide-and-conquer, randomization, and incremental construction all lead to popular sorting algorithms.,ùƒ' € €€€‚ÿ)ì‚Aƒ& €€€‚ÿ&ƒgƒ# €€€‚ÿOAƒ¶ƒ0 0€>€Ȁçĝ怉€‚ÿu Fundamental Data Typesœ\gƒs„a ’€ž€‘€€ç%,ív€‰€‚€熺óՀ‰€‚€ççHú4€‰€‚€ç“ÁŠ€‰€‚ÿu Containers u Dictionaries u Binary Search Trees u Priority QueuesÒr¶ƒE…` €ä€Ȁç*Àß2€‰€‚€ç‹N摀‰€‚€çìÜìð€‰€‚€çMkóO€‰€‚ÿu Specialized Data Structures u Sorting u Applications of Sorting u Approaches to Sortingö„s„;†r ²€ €‘€€ç®ùù®€‰€‚€çˆ€‰€‚€çpm€‰€‚€çÑ€ ̀‰€‚€ç23+€‰€‚ÿu Data Structures u Incremental Insertion u Divide and Conquer u Randomization u Bucketing TechniquesëŠE…&‡a €€ȀçÞ«4€€‰€‚€çuªù(€‰€‚€çÖ8ˆ€‰€‚€ç7Ç瀉€‚ÿu War Story: Stripping Triangulations u War Story: Mystery of the Pyramids u War Story: String 'em Up u Exercises&;†L‡# €€€‚ÿ)&‡u‡& €€€‚ÿŸœL‡3‰ Ё‹€€‚çQ8ۀ†"€ ‰€çexÊ«€†"€ ‰€ç@n_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€毌4õ€†"€‰€æ°Œ4õ€†"€‰€æÜŒ4õ€†"€‰€€çQ8ۀ‰€€çexÊ«€‰€€ç@n_ɀ‰€€‚‚ÿ Next: Fundamental Data Types Up: Techniques Previous: Implementation Challenges AlgorithmsMon Jun 2 23:33:50 EDT 1997)u‡\‰& €€€‚ÿµf3‰ŠO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1\‰BŠ1Uÿÿÿÿÿÿÿÿ ÿÿÿÿBŠfŠ$ŠfŠ" €€€ ÿGBŠ­Š1žÿÿÿÿÿÿÿÿ­ŠêŠ‡ÌBreaking Problems Down=fŠêŠ# €4€€‚ÿBreaking Problems Down B=­Š,Ž ؁‹€€ç¥8ۀ†"€ ‰€çexÊ«€†"€ ‰€çln_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÓŒ4õ€†"€‰€æÔŒ4õ€†"€‰€æÕŒ4õ€†"€‰€€ç¥8ۀ‰€€çexÊ«€‰€€çln_ɀ‰€‚€‚€‚ÿ Next: Dynamic Programming Up: Techniques Previous: Implementation ChallengesBreaking Problems DownOne of the most powerful techniques for solving problems is to break them down into smaller, more easily solved pieces. Smaller problems are less overwhelming, and they permit us to focus on details that are lost when we are studying the entire problem. For example, whenever we can break the problem into smaller instances of the same type of problem, a recursive algorithm starts to become apparent.úÇêŠ2Á3 4€€€€€€€‚‚ÿTwo important algorithm design paradigms are based on breaking problems down into smaller problems. Dynamic programming typically removes one element from the problem, solves the smaller problem, and then uses the solution to this smaller problem to add back the element in the proper way. Divide and conquer typically splits the problem in half, solves each half, then stitches the halves back together to form,Ž2ÁfŠ a full solution.Both of these techniques are important to know about. Dynamic programming in particular is a misunderstood and underappreciated technique. To demonstrate its utility in practice, we present no fewer than three war stories where dynamic programming played the decisive role.W3,Ž‰Á$ €f€€‚ÿThe take-home lessons for this chapter include:æ2Á¢Ã3 4€Í€Ȁ€€‚€€‚ÿu Many objects have an inherent left-to-right ordering among their elements, such as characters in a string, elements of a permutation, points around a polygon, or leaves in a search tree. For any optimization problem on such left-to-right objects, dynamic programming will likely lead to an efficient algorithm to find the best solution.  u Without an inherent left-to-right ordering on the objects, dynamic programming is usually doomed to require exponential space and time.ýljÁŸÅ6 :€€Ȁ€‚€€‚€€‚ÿu Once you understand dynamic programming, it can be easier to work out such algorithms from scratch than to try to look them up.u The global optimum (found, for example, using dynamic programming) is often noticeably better than the solution found by typical heuristics. How important this improvement is depends upon your application, but it can never hurt.u Binary search and its variants are the quintessential divide-and-conquer algorithms.,¢ÃËÅ' € €€€‚ÿ)ŸÅôÅ& €€€‚ÿ&ËÅÆ# €€€‚ÿLôÅfÆ0 0€8€Ȁç˜U F€‰€‚ÿu Dynamic Programming¥Æ}Çr ²€K€‘€€çù㥀‰€‚€çZr€‰€‚€ç»!c€‰€‚€ç'€‰€‚€ç}.!€‰€‚ÿu Fibonacci numbers u The Partition Problem u Approximate String Matching u Longest Increasing Sequence u Minimum Weight Triangulation>ÍfÆ»Èq °€›€Ȁç)–Nv€‰€‚€çÀ”€‰€‚€ç!#~€‰€‚€炱 ݀‰€‚€çã?'<€‰€‚ÿu Limitations of Dynamic Programming u War Story: Evolution of the Lobster u War Story: What's Past is Prolog u War Story: Text Compression for Bar Codes u Divide and Conquer¢Q}Ç]ÉQ r€¢€‘€€çDÎ-›€‰€‚€ç¥\4ú€‰€‚€çë:Y€‰€‚ÿu Fast Exponentiation u Binary Search u Square and Other RootsB»ÈŸÉ0 0€$€ȀçgyAž€‰€‚ÿu Exercises&]ÉÅÉ# €€€‚ÿ)ŸÉîÉ& €€€‚ÿ»ºÅÉ©Ë Ё…€€‚ç¥8ۀ†"€ ‰€çexÊ«€†"€ ‰€çln_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÖŒ4õ€†"€‰€æ׌4õ€†"€‰€æØŒ4õ€†"€‰€€ç¥8ۀ‰€€çexÊ«€‰€€çln_ɀ‰€€‚‚ÿ Next: Dynamic Programming Up: Techniques Previous: Implementation Challenges AlgorithmsMon Jun 2 23:33:50 EDT 1997)îÉÒË& €€€‚ÿµf©Ë‡ÌO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÒËžÌ1UÿÿÿÿÿÿÿÿÿÿÿÿžÌÜÌ$‡ÌÜÌ" €€€ ÿAžÌÍ1wÿÿÿÿÿÿÿÿÍTÍÝAGraph Algorithms7ÜÌTÍ# €(€€‚ÿGraph Algorithms ©cÍ F Z‚Ù€€çÕ8ۀ†"€ ‰€çexÊ«€†"€ ‰€çcn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÙŒ4õ€†"€‰€æÚŒ4õ€†"€‰€æÛŒ4õ€†"€‰€€çÕ8ۀ‰€€çexÊ«€‰€€çcn_ɀ‰€‚€‚€‚€€€€€€€€€€€€€€çE’P€†"€‰€‚ÿ Next: The Friendship Graph Up: Techniques Previous: Implementation ChallengesGraph Algorithms A graph G=(V,E) consists of a set of vertices V together with a set E of vertex pairs or edges. Graphs are important because they can be used to represent essentially any relatTÍ ÜÌionship. For example, graphs can model a network of roads, with cities as vertices and roads between cities as edges, as shown in Figure . Electronic circuits can also be modeled as graphs, with junctions as vertices and components as edges.     ŸTÍÇ? L€€€†"€€€‚‚€€‚ÿ  Figure: Modeling road networks and electronic circuits as graphs  The key to understanding many algorithmic problems is to think of them in terms of graphs. Graph theory provides a language for talking about the properties of graphs, and it is amazing how often messy applied problems have a simple description and solution in terms of classical graph properties.  Designing truly novel graph algorithms is a very difficult task. The key to using graph algorithms effectively in applications lies in correctly modeling your problem as a standard graph property, so you can take advantage of existing algorithms. Becoming familiar with many different graph algorithmic problems is more important than understanding the details of particular graph algorithms, particularly since Part II of this book can point you to an implementation as soon as you know the name of your problem.í Ù% €Û€€‚ÿIn this chapter, we will present basic data structures and traversal operations for graphs, which will enable you to cobble together solutions to rudimentary graph problems. We will also describe more sophisticated algorithms for problems like shortest paths and minimum spanning trees in some detail. But we stress the primary importance of correctly modeling your problem. Time spent browsing through the catalog now will leave you better informed of your options when a real job arises.V2Ç/$ €d€€‚ÿThe take-home lessons of this chapter include:ãÙL : B€Ç€Ȁ€€‚€€‚€€‚ÿu Graphs can be used to model a wide variety of structures and relationships.u Properly formulated, most applications of graphs can be reduced to standard graph properties and using well-known algorithms. These include minimum spanning trees, shortest paths, and several problems presented in the catalog.u Breadth-first and depth-first search provide mechanisms to visit each edge and vertex of the graph. They prove the basis of most simple, efficient graph algorithms.,/x ' € €€€‚ÿ)L ¡ & €€€‚ÿ&x Ç # €€€‚ÿအ š a €€ȀçÈH€‰€‚€çt€hl€‰€‚€ç -€‰€‚€çl 4t€‰€‚ÿu The Friendship Graph u Data Structures for Graphs u War Story: Getting the Graph u Traversing a Graphy8Ç ! A R€p€‘€€ç͛:Ӏ‰€‚€ç.*A2€‰€‚ÿu Breadth-First Search u Depth-First SearchX(š y 0 0€P€Ȁ珞G‘€‰€‚ÿu Applications of Graph Traversal”!  r ²€)€‘€€çðFNð€‰€‚€çQÕTO€‰€‚€ç²c[®€‰€‚€çòa €‰€‚€ç¿j‚b€‰€‚ÿu Connected Components u Tree and Cycle Detection u Two-Coloring Graphs u Topological Sorting u Articulation Vertices?y þ @ P€~€ȀçViG €‰€‚€ç·÷Mj€‰€‚ÿu Modeling Graph Problems u Minimum Spanning Treesv5 t A R€j€‘€€ç†Tɀ‰€‚€çy[(€‰€‚ÿu Prim's Algorithm u Kruskal's AlgorithmGþ » 0 0€.€ȀçÚ¢a‡€‰€‚ÿu Shortest Paths~=t 9A R€z€‘€€ç;1h怉€‚€眿nE€‰€‚ÿu Dijkstra's Algorithm u All-Pairs Shortest Path¯_» èP p€Ÿ€ȀçýMu€€‰€‚€ç^Ü{€‰€‚€ç UœX€‰€‚ÿu War Story: Nothing but Nets u War Story: Dialing for Documents u Exercises&9# €€€‚ÿ)è7& €€€‚ÿŒ»ÿ@ Ё‡€€‚çÕ8ۀ†"€ ‰€çexÊ«€†"€ ‰€çcn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æœ4õ€†"€‰€æþŒ4õ€†"€‰€æÿŒ4õ€†"€‰€7ÿ@ÜÌ€çÕ8ۀ‰€€çexÊ«€‰€€çcn_ɀ‰€€‚‚ÿ Next: The Friendship Graph Up: Techniques Previous: Implementation Challenges AlgorithmsMon Jun 2 23:33:50 EDT 1997)7(A& €€€‚ÿµfÿ@ÝAO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1(AB1UÿÿÿÿÿÿÿÿÿÿÿÿB2B$ÝA2B" €€€ ÿ[*BB1ÿÿÿÿÿÿÿÿBÞB¶„Combinatorial Search and Heuristic MethodsQ.2BÞB# €\€€‚ÿCombinatorial Search and Heuristic Methods Q.B/F# ‚q€€çN9ۀ†"€ ‰€çexÊ«€†"€ ‰€çdn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æœ4õ€†"€‰€æœ4õ€†"€‰€æœ4õ€†"€‰€€çN9ۀ‰€€çexÊ«€‰€€çdn_ɀ‰€‚€‚€†"€çBה€†"€‰€‚ÿ Next: Backtracking Up: Techniques Previous: Implementation ChallengesCombinatorial Search and Heuristic MethodsWe have seen how clever algorithms can reduce the complexity of sorting from to , which is good. However, the algorithmic stakes can be even higher for combinatorially explosive problems, whose time grows exponentially in the size of the problem. Looking back at Figure will make clear the limitations of exponential-time algorithms on even modest-sized problems.  qLÞB H% €™€€‚ÿBy using exhaustive search techniques, we can solve small problems to optimality, although the time complexity may be enormous. For certain applications, it may well pay to spend extra time to be certain of the optimal solution. A good example occurs in testing a circuit or a program on all possible inputs. You can prove the correctness of the device by trying all possible inputs and verifying that they give the correct answer. Proving such correctness is a property to be proud of. However, claiming that it works correctly on all the inputs you tried is worth much, much less.  ™t/F9K% €é€€‚ÿIn this section, we present backtracking as a technique for listing all configurations representing possible solutions for a combinatorial algorithm problem. We then discuss techniques for pruning search that significantly improve efficiency by eliminating irrelevant configurations from consideration. We illustrate the power of clever pruning techniques to speed up real search applications. For problems that are too large to contemplate using brute-force combinatorial search, we introduce heuristic methods such as simulated annealing. Such heuristic methods are an important weapon in the practical algorist's arsenal.T0 HK$ €`€€‚ÿThe take-home lessons from this chapter are:j09K÷M: B€a€Ȁ€€‚€€‚€€‚ÿu Combinatorial search, augmented with tree pruning techniques, can be used to find the optimal solution of small optimization problems. How small depends upon the specific problem, but the size limit is likely to be somewhere between items.u Clever pruning techniques can speed up combinatorial search to an amazing extent. Proper pruning will have a greater impact on search time than any other factor.u Simulated annealing is a simple but effective technique to efficiently obtain good but not optimal solutions to combinatorial search problems.,K#N' € €€€‚ÿ)÷MLN& €€€‚ÿ&#NrN# €€€‚ÿELN·N0 0€*€Ȁç¡Sa€‰€‚ÿu BacktrackingÂqrNyOQ r€â€‘€€çâg`€‰€‚€çcpn¿€‰€‚€çÄþt€‰€‚ÿu Constructing All Subsets u Constructing All Permutations u Constructing All Paths in a GraphØx·N]€` €ð€Ȁç%{}€‰€‚€ç†‚Ü€‰€‚€ç穈;€‰€‚€çH8š€‰€‚ÿu Search Pruning u Bandwidth MyO]€2Binimization u War Story: Covering Chessboards u Heuristic Methods OyOý€Q r€ž€‘€€ç©Æ•ù€‰€‚€çL€‚€ç­žÉ €‰€‚ÿu Simulated Annealing u Neural Networks u Genetic AlgorithmsØx]€Ձ` €ð€Ȁç-Ðl€‰€‚€ço»Öˀ‰€‚€çÐIÝ*€‰€‚€ç1Ø㉀‰€‚ÿu War Story: Annealing Arrays u Parallel Algorithms u War Story: Going Nowhere Fast u Exercises&ý€û# €€€‚ÿ)Ձ$‚& €€€‚ÿŽ³û؃ Ёw€€‚çN9ۀ†"€ ‰€çexÊ«€†"€ ‰€çdn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æœ4õ€†"€‰€æœ4õ€†"€‰€æœ4õ€†"€‰€€çN9ۀ‰€€çexÊ«€‰€€çdn_ɀ‰€€‚‚ÿ Next: Backtracking Up: Techniques Previous: Implementation Challenges AlgorithmsMon Jun 2 23:33:50 EDT 1997)$‚„& €€€‚ÿµf؃¶„O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1„ç„1Uÿÿÿÿÿÿÿÿÿÿÿÿç„ …$¶„ …" €€€ ÿX'ç„c…1£ÿÿÿÿÿÿÿÿc…±…ŠÉIntractable Problems and ApproximationsN+ …±…# €V€€‚ÿIntractable Problems and Approximations h[c…‰  èÇ€€ç\f_ɀ†"€ ‰€çexÊ«€†"€ ‰€çen_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æœ4õ€†"€‰€æ2œ4õ€†"€‰€æ)œ4õ€†"€‰€€ç\f_ɀ‰€€çexÊ«€‰€€çen_ɀ‰€‚€‚€‚€€‚ÿ Next: Problems and Reductions Up: Techniques Previous: Implementation ChallengesIntractable Problems and Approximations In this chapter, we will concentrate on techniques for proving that no efficient algorithm exists for a given problem. The practical reader is probably squirming at the notion of proving anything and will be particularly alarmed at the idea of investing time to prove that something does not exist. Why will you be better off knowing that something you don't know how to do in fact can't be done at all?  ã±…'Œ+ $€Ç€€€€‚ÿThe truth is that the theory of NP-completeness is an immensely useful tool for the algorithm designer, even though all it does is provide negative results. That noted algorithm designer Sherlock Holmes once said, ``When you have eliminated the impossible, what remains, however improbable, must be the truth.'' The theory of NP-completeness enables the algorithm designer to focus her efforts more productively, by revealing that the search for an efficient algorithm for this particular problem is doomed to failure. When one fails to show that a problem is hard, that means there is likely an algorithm that solves it efficiently. Two of the war stories in this book describe happy results springing from bogus claims of hardness.Ú‰:9 @€µ€€‚€€çՈK€‰€‚ÿThe theory of NP-completeness also enables us to identify exactly what properties make a particular problem hard, thus providing direction for us to model it in different ways or exploit more benevolent characteristics of the problem. Developing a sense for which problems are hard and which are not is a fundamental skill for algorithm designers, and it can come only from hands-on experience proving hardness.We will not discuss the complexity-theoretic aspects of NP-completeness in depth, limiting our treatment to the fundamental concept of reductions, which show the equivalence of pairs of problems. For a discussion, we refer the reader to [GJ79], the truly essential reference on the theory of intractability.T0'ŒŽ$ €`€€‚ÿThe take-home lessons from this chapter are:+ñ:ÅÁ: B€ã€Ȁ€€‚€€‚€€‚ÿu Reductions are a way to show that two problems areŽÅÁ … essentially identical. A fast algorithm for one of the problems implies a fast algorithm for the other.u In practice, a small set of NP-complete problems (3-SAT, vertex cover, integer partition, and Hamiltonian cycle) suffice to prove the hardness of most other hard problems.u Approximation algorithms guarantee answers that are always close to the optimal solution and can provide an approach to dealing with NP-complete problems.,ŽñÁ' € €€€‚ÿ)ÅÁÂ& €€€‚ÿ&ñÁ@Â# €€€‚ÿz:ºÂ@ P€t€Ȁçï)B€‰€‚€ç†(Æꀉ€‚ÿu Problems and Reductions u Simple Reductionsžg@ÂrÃQ r€Î€‘€€çç¶ÌI€‰€‚€çHEÓš€‰€‚€ç©ÓÙ€‰€‚ÿu Hamiltonian Cycles u Independent Set and Vertex Cover u Clique and Independent SetGºÂ¹Ã0 0€.€Ȁç bàf€‰€‚ÿu Satisfiability€?rÃ9ÄA R€~€‘€€çkðæŀ‰€‚€çÌ~í$€‰€‚ÿu The Theory of NP-Completeness u 3-SatisfiabilityM¹Ã†Ä0 0€:€Ȁç- ôƒ€‰€‚ÿu Difficult Reductionsr19ÄøÄA R€b€‘€€玛ú €‚€ç:8€‰€‚ÿu Integer Programming u Vertex Coveró’†ÄëÅa €%€ȀçÑàà€‰€‚€ç2¡æ?€‰€‚€ç“/힀‰€‚€çôœóý€‰€‚ÿu Other NP-Complete Problems u The Art of Proving Hardness u War Story: Hard Against the Clock u Approximation AlgorithmsLøÄxÆA R€˜€‘€€çULú\€‰€‚€ç¶ÚŒ€‰€‚ÿu Approximating Vertex Cover u The Euclidean Traveling SalesmanBëźÆ0 0€$€Ȁçi€‰€‚ÿu Exercises&xÆàÆ# €€€‚ÿ)ºÆ Ç& €€€‚ÿ¿ŸàÆÈÈ Ё€€‚ç\f_ɀ†"€ ‰€çexÊ«€†"€ ‰€çen_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ*œ4õ€†"€‰€æ+œ4õ€†"€‰€æ,œ4õ€†"€‰€€ç\f_ɀ‰€€çexÊ«€‰€€çen_ɀ‰€€‚‚ÿ Next: Problems and Reductions Up: Techniques Previous: Implementation Challenges AlgorithmsMon Jun 2 23:33:50 EDT 1997) ÇñÈ& €€€‚ÿµfÈÈŠÉO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ñÈ×É1Uÿÿÿÿÿÿÿÿÿÿÿÿ×ÉûÉ$ŠÉûÉ" €€€ ÿI×ÉDÊ1Š2ÿÿÿÿÿÿÿÿDʃÊیHow to Design Algorithms?ûɃÊ# €8€€‚ÿHow to Design Algorithms FADÊÉÍ ؁“€€çú7ۀ†"€ ‰€çexÊ«€†"€ ‰€çfn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ-œ4õ€†"€‰€æ.œ4õ€†"€‰€æ/œ4õ€†"€‰€€çú7ۀ‰€€çexÊ«€‰€€çfn_ɀ‰€‚€‚€‚ÿ Next: Resources Up: Techniques Previous: Implementation ChallengesHow to Design AlgorithmsDesigning the right algorithm for a given application is a difficult job. It requires a major creative act, taking a problem and pulling a solution out of the ether. This is much more difficult than taking someone else's idea and modifying it or tweaking it to make it a little better. The space of choices you can make in algorithm design is enormous, enough to leave you plenty of freedom to hang yourself.     uPƒÊJ% €¡€€‚ÿThis book is designed to make you a better algorithm designer. The techniques presented in Part I of this book provide the basic ideas underlying all combinatorial algorithms. The problem catalog of Part II will help you with modeling your application and point you in the right direction of an algorithm or implementation. However, being a successful algorithm designer requires more than book knowledge; it requires a certain attitude, the right problem-solving approach. It is difficult to teach this mindset in a book; yet gÉÍJûÉetting it is essential to become a successful designer.     >É͈% €3€€‚ÿThe key to algorithm design (or any other problem-solving task) is to proceed by asking yourself a sequence of questions to guide your thought process. What if we do this? What if we do that? Should you get stuck on the problem, the best thing to do is move onto the next question. In any group brainstorming session, the most useful person in the room is the one who keeps asking, ``Why can't we do it this way?'' not the person who later tells them why. Because eventually she will stumble on an approach that can't be shot down.  öÑJ~% €£€€‚ÿTowards this end, we provide below a sequence of questions to guide your search for the right algorithm for your problem. To use it effectively, you must not only ask the questions, but answer them. The key is working through the answers carefully, by writing them down in a log. The correct answer to, ``Can I do it this way?'' is never ``no,'' but ``no, because ....'' By clearly articulating your reasoning as to why something doesn't work, you can check if it really holds up or whether you have just glossed over a possibility that you didn't want to think hard enough about. You will be surprised how often the reason you can't find a convincing explanation for something is because your conclusion is wrong.   Õ€ˆS 1 0€I€€€€€€‚ÿAn important distinction to keep aware of during any design process is the difference between strategy and tactics. Strategy represents the quest for the big picture, the framework around which we construct our path to the goal. Tactics are used to win the minor battles we must fight along the way. In problem solving, it is important to check repeatedly whether you are thinking on the right level. If you do not have a global strategy of how you are going to attack your problem, it is pointless to worry about the tactics. An example of a strategic question is, ``How best can I model my application as a graph algorithm problem?'' A tactical question might be, ``Should I use an adjacency list or adjacency matrix data structure to represent my graph?'' Of course, such tactical decisions are critical to the ultimate quality of the solution, but they can be properly evaluated only in light of a successful strategy.  ë~q 3 4€×€€€€€€‚‚ÿWhen faced with a design problem, too many people freeze up in their thinking. After reading or hearing the problem, they sit down and realize that they don't know what to do next. They stare into space, then panic, and finally end up settling for the first thing that comes to mind. Avoid this fate. Follow the sequence of questions provided below and in most of the catalog problem sections. We'll tell you what to do next!Obviously, the more experience you have with algorithm design techniques such as dynamic programming, graph algorithms, intractability, and data structures, the more successful you will be at working through the list of questions. Part I of this book has been designed to strengthen this technical background. However, it pays to work through these questions regardless of how strong your technical skills are. The earliest and most important questions on the list focus on obtaining a detailed understanding of the problem and do not require specific expertise.      ÒS O@C T€€€€€簞K€‰€€€€€‚ÿThis list of questions was inspired by a passage in that wonderful book about the space program The Right Stuff [Wol79]. It concerned the radio transmissions from test pilots just before their planes crashed. One might have expected that they would panic, so that ground control would hear the pilot yelling Ahhhhhhhhhhh --, terminated only by the sound of smacking into a mountain. Instead, the pilots ran through a list of what their possible actions could be. I've tried the flaps. I've checked the engine. Still got two wings. I've reset the --. They had ``the Right Stuffq O@ûÉ.'' Because of this, they sometimes managed to miss the mountain.ײq &A% €e€€‚ÿI hope this book and list will provide you with ``the Right Stuff'' to be an algorithm designer. And I hope it prevents you from smacking into any mountains along the way.   O*O@uA% €T€Ȁ‚ÿ1. Do I really understand the problem?qE&AæC, &€‹€‘€€‚‚‚‚‚‚ÿ1. What exactly does the input consist of?2. What exactly are the desired results or output?3. Can I construct an example input small enough to solve by hand? What happens when I try to solve it?4. How important is it to my application that I always find an exact, optimal answer? Can I settle for something that is usually pretty good?5. How large will a typical instance of my problem be? Will I be working on 10 items? 1,000 items? 1,000,000 items?6. How important is speed in my application? Must the problem be solved within one second? One minute? One hour? One day?äœuAÊE' €{€‘€€‚‚ÿ7. How much time and effort can I invest in implementing my algorithm? Will I be limited to simple algorithms that can be coded up in a day, or do I have the freedom to experiment with a couple of approaches and see which is best?8. Am I trying to solve a numerical problem? A graph algorithm problem? A geometric problem? A string problem? A set problem? Might my problem be formulated in more than one way? Which formulation seems easiest?gBæC1F% €„€Ȁ‚ÿ2. Can I find a simple algorithm or heuristic for the problem?¶‰ÊEçF- (€€‘€€€€‚ÿ1. Can I find an algorithm to solve my problem correctly by searching through all subsets or arrangements and picking the best one?êº1FÑH0 .€u€ـ€‚‚‚€€‚ÿ1. If so, why am I sure that this algorithm always gives the correct answer?2. How do I measure the quality of a solution once I construct it?3. Does this simple, slow solution run in polynomial or exponential time? Is my problem small enough that this brute-force solution will suffice?4. If I can't find a slow, guaranteed correct algorithm, why am I certain that my problem is sufficiently well-defined to have a correct solution?Œ–çFI& €-€‘€€‚ÿ2. Can I solve my problem by repeatedly trying some simple rule, like picking the biggest item first? The smallest item first? A random item first?gÑHK) €Ï€ـ€‚‚‚ÿ1. If so, on what types of inputs does this heuristic work well? Do these correspond to the data that might arise in my application?2. On what types of inputs does this heuristic work badly? If no such examples can be found, can I show that it always works well?3. How fast does my heuristic come up with an answer? Does it have a simple implementation?zUI—K% €ª€Ȁ‚ÿ3. Is my problem in the catalog of algorithmic problems in the back of this book?Î¥KeM) €K€‘€€‚‚‚ÿ1. If it is, what is known about the problem? Is there an implementation available that I can use?2. If I don't see my problem, did I look in the right place? Did I browse through all the pictures? Did I look in the index under all possible keywords?3. Are there relevant resources available on the World-Wide Web? Did I do a Lycos, Alta Vista, or Yahoo search? Did I go to the WWW page associated with this book, ?tO—KÙM% €ž€Ȁ‚ÿ4. Are there special cases of the problem that I know how to solve exactly?CeM(€1 0€%€‘€€‚‚€€‚‚‚ÿ1. Can I solve the problem efficiently when I ignore some of the input parameters?2. What happens when I set some of the input parameters to trivial values, such as 0 or 1? Does the problem become easier to solve?3. Can I simplify the problem to the point where I can solve it efficiently? Is the problem now trivial or still interesting?4. Once I know how to solve a certain special case, why can't this be generalized to a wider class of inputs?5. Is my problem a special case of a more generÙM(€ûÉal problem in the catalog?}XÙM¥€% €°€Ȁ‚ÿ5. Which of the standard algorithm design paradigms are most relevant to my problem?h?(€ ƒ) €€‘€€‚‚‚ÿ1. Is there a set of items that can be sorted by size or some key? Does this sorted order make it easier to find the answer?2. Is there a way to split the problem in two smaller problems, perhaps by doing a binary search? How about partitioning the elements into big and small, or left and right? Does this suggest a divide-and-conquer algorithm?3. Do the input objects or desired solution have a natural left-to-right order, such as characters in a string, elements of a permutation, or the leaves of a tree? If so, can I use dynamic programming to exploit this order?B¥€O…' €7€‘€€‚‚ÿ4. Are there certain operations being repeatedly done on the same data, such as searching it for some element, or finding the largest/smallest remaining element? If so, can I use a data structure to speed up these queries? What about a dictionary/hash table or a heap/priority queue?5. Can I use random sampling to select which object to pick next? What about constructing many random configurations and picking the best one? Can I use some kind of directed randomness like simulated annealing in order to zoom in on the best solution?™f ƒè†3 4€Í€‘€€‚çՈK€‰€‚ÿ6. Can I formulate my problem as a linear program? How about an integer program?7. Does my problem seem something like satisfiability, the traveling salesman problem, or some other NP-complete problem? If so, might the problem be NP-complete and thus not have an efficient algorithm? Is it in the problem list in the back of Garey and Johnson [GJ79]??O…'‡% €4€Ȁ‚ÿ6. Am I still stumped?q4膘ˆ= H€k€‘€€çY„€†"€‰€‚‚ÿ1. Am I willing to spend money to hire an expert to tell me what to do? If so, check out the professional consulting services mentioned in Section .  2. Why don't I go back to the beginning and work through these questions again? Did any of my answers change during my latest trip through the list?‹Q'‡#Š: B€£€€€€çϘK€‰€€‚ÿProblem solving is not a science, but part art and part skill. It is one of the skills most worth developing. My favorite book on problem solving remains Pólya's How to Solve It [Pol57], which features a catalog of problem solving techniques that are fascinating to browse through, both before and after you have a problem.   )˜ˆLŠ& €€€‚ÿ±°#Šý‹ Ёq€€‚çú7ۀ†"€ ‰€çexÊ«€†"€ ‰€çfn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ0œ4õ€†"€‰€æ1œ4õ€†"€‰€æ]œ4õ€†"€‰€€çú7ۀ‰€€çexÊ«€‰€€çfn_ɀ‰€€‚‚ÿ Next: Resources Up: Techniques Previous: Implementation Challenges AlgorithmsMon Jun 2 23:33:50 EDT 1997)LŠ&Œ& €€€‚ÿµfý‹یO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1&Œ 1Uÿÿÿÿÿÿÿÿÿÿÿÿ 0$ی0" €€€ ÿ: j1-ÿÿÿÿÿÿÿÿjš@ÄResources0 0š# €€€‚ÿResources œµjW ށ{€€çû7ۀ†"€ ‰€çhxÊ«€†"€ ‰€çù7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æTœ4õ€†"€‰€æUœ4õ€†"€‰€æVœ4õ€†"€‰€€çû7ۀ‰€€çhxÊ«€‰€€çù7ۀ‰€‚€‚€€‚ÿ Next: A Catalog of Algorithmic Up: The Algorithm Design Manual Previous: How to Design AlgorithmsResources)š€& €€€‚ÿ&WŠ# €€€‚ÿZ*€ À0 0€T€Ȁç.ɲ+€‰€‚ÿu A Catalog of Algorithmic ProblemsŠ À0qߊ}Á’ ò€¿€‘€€çx÷ z€‰€‚€ç 6S€‰€‚€ç­=;€‰€‚€çY¶[\€‰€‚€çHŠa¯€‰€‚€çU­ˆc€‰€‚€ç+ÁD?€‰€‚ÿu Data Structures u Numerical Problems u Combinatorial Problems u Graph Problems: Polynomial-Time u Graph Problems: Hard Problems u Computational Geometry u Set and String ProblemsN ÀËÁ0 0€<€ȀçW¹Š€‰€‚ÿu Algorithmic Resourcesõƒ}ÁÀÂr ²€€‘€€爄[˜€‰€‚€çNT€‰€‚€çîŠT³€‰€‚€çO5[€‰€‚€çY„€‰€‚ÿu Software systems u Data Sources u Textbooks u On-Line Resources u Professional Consulting Services&ËÁæÂ# €€€‚ÿ)ÀÂÃ& €€€‚ÿS+æÂbÃ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)ËÃ& €€€‚ÿµfbÃ@ÄO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1‹ÃqÄ1UÿÿÿÿÿÿÿÿÿÿÿÿqÄ•Ä$@Ä•Ä" €€€ ÿR!qÄçÄ1q&ÿÿÿÿÿÿÿÿçÄ/ÅRJA Catalog of Algorithmic ProblemsH%•Ä/Å# €J€€‚ÿA Catalog of Algorithmic Problems ohçÄžÇ ܁ကçgn_ɀ†"€ ‰€çú7ۀ†"€ ‰€çú7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æWœ4õ€†"€‰€æXœ4õ€†"€‰€æYœ4õ€†"€‰€€çgn_ɀ‰€€çú7ۀ‰€€çú7ۀ‰€‚€‚€‚‚ÿ Next: Caveats Up: Resources Previous: ResourcesA Catalog of Algorithmic Problems This is a catalog of algorithmic problems that arise commonly in practice. It describes what is known about them and gives suggestions about how best to proceed if the problem arises in your application.I/ÅçÊ1 0€1€€€€€€‚ÿWhat is the best way to use this catalog? First, think a little about your problem. If you recall the name of your problem, look up the catalog entry in the index or table of contents and start reading. Read through the entire entry, since it contains pointers to other relevant problems that might be yours. If you don't find what you are looking for, leaf through the catalog, looking at the pictures and problem names to see if something strikes a chord. Don't be afraid to use the index, for every problem in the book is listed there under several possible keywords and applications.   If you still don't find something relevant, your problem is either not suitable for attack by combinatorial algorithms or else you don't fully understand it. In either case, go back to step one. ùžÇÏ' €ó€€‚‚ÿThe catalog entries contain a variety of different types of information that have never been collected in one place before. Different fields in each entry present information of practical and historical interest.To make this catalog more easily accessible, we introduce each problem with a pair of graphics representing the problem instance or input on the left and the result of solving the problem on this instance on the right.   We have invested considerable thought in selecting stylized images and examples that illustrate desired behaviors, more than just definitions. For example, the minimum spanning tree example illustrates how points can be clustered using minimum spanning trees. We hope that people without a handle on algorithmic terminology can flip through the pictures and identify which problems might be relevant to them. We augment these pictures with more formal written input and problem descriptions in order to eliminate any ambiguity inherent in a purely pictorial representation.     >çÊQ% €3€€‚ÿOnce you have identified your problem of interest, the discussion section tells you what you should do about it.   We describe applications where the problem is likely to arise and special issues associated witÏQ•Äh data from them. We discuss the kind of results you can hope for or expect and, most importantly, what you should do to get them. For each problem, we outline a quick-and-dirty algorithm and pointers to algorithms to try next if the first attempt is not sufficient. We also identify other, related problems in the catalog.y6ÏÊC T€q€€ç†W¹Š€†"€‰€†"€‚ÿFor most if not all of the problems presented, we identify readily available software implementations, which are discussed   in the implementation field of each entry. Many of these routines are quite good, and they can perhaps be plugged directly into your application. Others will be incomplete or inadequate for production use, but they hopefully can provide a good model for your own implementation. In general, the implementations are listed in order of descending usefulness, but we will explicitly recommend the best one available for each problem if a clear winner exists. More detailed information for many of these implementations appears in Chapter .     Essentially all of the implementations are available via the WWW site associated with this book, reachable at http://www.cs.sunysb.edu/ algorith.ôÌQŸ( €™€€€‚ÿFinally, in deliberately smaller print, we discuss the history of each problem and present results of primarily theoretical interest.   We have attempted to report the best results known for each problem and point out empirical comparisons of algorithms if they exist. This should be of interest to students and researchers, and also to practitioners for whom our recommended solutions prove inadequate and who need to know if anything better is possible.)Êç& €€€‚ÿ&Ÿ # €€€‚ÿHçU0 0€0€Ȁçx÷ z€‰€‚ÿu Data Structures˜ o‚ Ҁ1€‘€€çمـ‰€‚€ç…þ4.€‰€‚€çýùր‰€‚€ç}‹6€‰€‚€çÞ•€‰€‚€ç?š ô€‰€‚ÿu Dictionaries u Priority Queues u Suffix Trees and Arrays u Graph Data Structures u Set Data Structures u Kd-TreesKUº0 0€6€Ȁç 6S€‰€‚ÿu Numerical ProblemsWoÓ  R¯€‘€€çŲ€‰€‚€çbS!€‰€‚€çÃá'p€‰€‚€ç$p.π‰€‚€çÐèN$€‰€‚€çgç̀‰€‚€çÈu,€‰€‚€ç)!‹€‰€‚€犒'ꀉ€‚€çë .I€‰€‚ÿu Solving Linear Equations u Bandwidth Reduction u Matrix Multiplication u Determinants and Permanents u Constrained and Unconstrained Optimization u Linear Programming u Random Number Generation u Factoring and Primality Testing u Arbitrary-Precision Arithmetic u Knapsack Problem T#º' 1 2€F€‘€€çL¯4š€‰€‚ÿu Discrete Fourier TransformOÓ v 0 0€>€Ȁç­=;€‰€‚ÿu Combinatorial ProblemsÃ' 9  R€‘€€çÌAf€‰€‚€çoZHŀ‰€‚€çÓh€‰€‚€ç²Ñ-À‰€‚€ç`4"€‰€‚€çtî:€‰€‚€çÕ|Aà€‰€‚€ç6 H?€‰€‚€痙Nž€‰€‚€çø'Uý€‰€‚ÿu Sorting u Searching u Median and Selection u Generating Permutations u Generating Subsets u Generating Partitions u Generating Graphs u Calendrical Calculations u Job Scheduling u SatisfiabilityX(v ‘ 0 0€P€ȀçY¶[\€‰€‚ÿu Graph Problems: Polynomial-Time?9 ¢Ò r€‘€€çºDb»€‰€‚€çfœ‚€‰€‚€çý»G¹€‰€‚€ç^JN€‰€‚€ç¿ØTw€‰€‚€ç g[ր‰€‚€çõa5€‰€‚€çâƒh”€‰€‚€çCo󀉀‚€瀠uR€‰€‚€ç/|±€‰€‚ÿu Connected Components u Topological Sorting u Minimum Spanning Tree u Shortest Path u Transitive Closure and Reduction u Matching u Eulerian Cycle / Chinese Postman u Edge and Vertex Connectivity u Network Flow u Drawing Graphs Nicely u Drawing Trees [*‘  @1 2€T€‘€€籧œ€‰€‚ÿu Planarity Detection and Embedding¢ @•ÄV&¢b@0 0€L€ȀçHŠa¯€‰€‚ÿu Graph Problems: Hard Problemsá @CBÒ r€‘€€ç©4h€‰€‚€ç Ãnm€‰€‚€çkQù‰€‚€çÌß{+€‰€‚€ç-n‚Š€‰€‚€çŽüˆé€‰€‚€çH€‰€‚€çP–§€‰€‚€çü‘¶ü€‰€‚€瓐{¥€‰€‚€çô‚€‰€‚ÿu Clique u Independent Set u Vertex Cover u Traveling Salesman Problem u Hamiltonian Cycle u Graph Partition u Vertex Coloring u Edge Coloring u Graph Isomorphism u Steiner Tree u Feedback Edge/Vertex SetOb@’B0 0€>€ȀçU­ˆc€‰€‚ÿu Computational Geometry&DCBžD⠒‰€‘€€ç¶;Â€‰€‚€çʕ!€‰€‚€çxXœ€€‰€‚€çÙæ¢ß€‰€‚€ç:u©>€‰€‚€ç›°€‰€‚€çG|Ðò€‰€‚€çÞz•›€‰€‚€ç? œú€‰€‚€砗¢Y€‰€‚€ç&©ž€‰€‚€çbŽ¯€‰€‚ÿu Robust Geometric Primitives u Convex Hull u Triangulation u Voronoi Diagrams u Nearest Neighbor Search u Range Search u Point Location u Intersection Detection u Bin Packing u Medial-Axis Transformation u Polygon Partitioning u Simplifying Polygons Îm’B†Ea ’€Ú€‘€€çÃB¶v€‰€‚€ç$ьՀ‰€‚€ç…_Ã4€‰€‚€çæíɓ€‰€‚ÿu Shape Similarity u Motion Planning u Maintaining Line Arrangements u Minkowski SumP žDÖE0 0€@€Ȁç+ÁD?€‰€‚ÿu Set and String Problems±ÿ†E‡G² 2ÿ€‘€€ç¿ 耉€‚€ç#NG€‰€‚€ç„ÜŠ€‰€‚€çåj€‰€‚€çFù#d€‰€‚€秇*À‰€‚€ç1"€‰€‚€çi€7€‰€‚€çÊ2>à€‰€‚ÿu Set Cover u Set Packing u String Matching u Approximate String Matching u Text Compression u Cryptography u Finite State Machine Minimization u Longest Common Substring u Shortest Common Superstring&ÖE­G# €€€‚ÿ)‡GÖG& €€€‚ÿž­GtI ЁK€€‚çgn_ɀ†"€ ‰€çú7ۀ†"€ ‰€çú7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æZœ4õ€†"€‰€æ[œ4õ€†"€‰€æ\œ4õ€†"€‰€€çgn_ɀ‰€€çú7ۀ‰€€çú7ۀ‰€€‚‚ÿ Next: Caveats Up: Resources Previous: Resources AlgorithmsMon Jun 2 23:33:50 EDT 1997)ÖGI& €€€‚ÿµftIRJO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1IƒJ1UÿÿÿÿÿÿÿÿÿÿÿÿƒJ§J$RJ§J" €€€ ÿFƒJíJ1/ÿÿÿÿÿÿÿÿíJ)K0ƒAlgorithmic Resources<§J)K# €2€€‚ÿAlgorithmic Resources ]SíJ†N  ⁷€€ç•m_ɀ†"€ ‰€çú7ۀ†"€ ‰€ço_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€戜4õ€†"€‰€æœ4õ€†"€‰€怜4õ€†"€‰€€ç•m_ɀ‰€€çú7ۀ‰€€ço_ɀ‰€‚€‚€‚€‚ÿ Next: Software systems Up: Resources Previous: Shortest Common SuperstringAlgorithmic Resources This chapter describes resources that the practical algorithm designer should be familiar with. Although some of this information has appeared at various points in the catalog, the most important pointers have been collected here for general reference. These resources take the form of software systems and libraries, books, and other bibliographic sources. Many of the most interesting resources are available on-line.   ))K¯N& €€€‚ÿ&†NÕN# €€€‚ÿI¯NO0 0€2€Ȁ爄[˜€‰€‚ÿu Software systems–ÕNB€‚ Ҁ-€‘€€çƒ A€‰€‚€ç€' €‰€‚€çB.4^€‰€‚€磌:œ€‰€‚€çKA€‰€‚€çeÙG{€‰€‚ÿu LEDA u Netlib u The Stanford GraphBase u Combinatorica u Algorithm OB€§JAnimations with XTango u Programs from Books‘AOӀP p€‚€ȀçNT€‰€‚€çîŠT³€‰€‚€çO5[€‰€‚ÿu Data Sources u Textbooks u On-Line Resources„3B€WQ r€f€‘€€ç°Ãaq€‰€‚€çRhЀ‰€‚€çràn/€‰€‚ÿu Literature u People u SoftwareY)Ӏ°0 0€R€ȀçY„€‰€‚ÿu Professional Consulting Services&Wց# €€€‚ÿ)°ÿ& €€€‚ÿS+ցR‚( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)ÿ{‚& €€€‚ÿµfR‚0ƒO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1{‚aƒ1Uÿÿÿÿÿÿÿÿÿÿÿÿaƒ…ƒ$0ƒ…ƒ" €€€ ÿ; aƒÀƒ1ÏÿÿÿÿÿÿÿÿÀƒñƒÁ References1…ƒñƒ# €€€‚ÿReferences ³®Àƒ€… ؁m€€çbxÊ«€†"€ ‰€çhxÊ«€†"€ ‰€çëm_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€恜4õ€†"€‰€æ‚œ4õ€†"€‰€惜4õ€†"€‰€€çbxÊ«€‰€€çhxÊ«€‰€€çëm_ɀ‰€‚€ ‚€‚ÿ Next: Index Up: The Algorithm Design Manual Previous: Professional Consulting ServicesReferencesAB95œ’ñƒa†+ $€%€Ȁ€€‚ÿD. Avis and D. Bremner. How good are convex hull algorithms? In Proc. 11th Annual ACM Symposium on Computational Geometry, pages 20-28, 1995.2 €…“†' €€€€‚ÿABCC95Ѭa†d‡% €Y€Ȁ‚ÿD. Applegate, R. Bixby, V. Chvatal, and W. Cook. Finding cuts in the TSP (a preliminary report). Technical Report 95-05, DIMACS, Rutgers University, Piscataway NJ, 1995.1 “†•‡' €€€€‚ÿAbd80ӟd‡hˆ4 6€A€Ȁ†"€€€‚ÿN. N. Abdelmalek. A Fortran subroutine for the solution of overdetermined systems of linear equations. ACM Trans. Math. Softw., 6(2):228-230, June 1980.0 •‡˜ˆ' €€€€‚ÿAC75¯„hˆG‰+ $€ €Ȁ€€‚ÿA. Aho and M. Corasick. Efficient string matching: an aid to bibliographic search. Communications of the ACM, 18:333-340, 1975.0 ˜ˆw‰' €€€€‚ÿAC91¯„G‰&Š+ $€ €Ȁ€€‚ÿD. Applegate and W. Cook. A computational study of the job-shop scheduling problem. ORSA Journal on Computing, 3:149-156, 1991.= w‰cŠ0 0€€€€†"€‚ÿACH 91åº&ŠH‹+ $€u€Ȁ€€‚ÿE. M. Arkin, L. P. Chew, D. P. Huttenlocher, K. Kedem, and J. S. B. Mitchell. An efficiently computable metric for comparing polygonal shapes. IEEE Trans. PAMI, 13(3):209-216, 1991.1 cŠy‹' €€€€‚ÿACI92Ú¯H‹SŒ+ $€_€Ȁ€€‚ÿD. Alberts, G. Cattaneo, and G. Italiano. An empirical study of dynamic graph algorithms. In Proc. Seventh ACM-SIAM Symp. Discrete Algorithms (SODA), pages 192-201, 1992.2 y‹…Œ' €€€€‚ÿADKF70ߎSŒd+ $€i€Ȁ€€‚ÿV. Arlazarov, E. Dinic, M. Kronrod, and I. Faradzev. On economical construction of the transitive closure of a directed graph. Soviet Mathematics, Doklady, 11:1209-1210, 1970.2 …Œ–' €€€€‚ÿAdl94aɞd_Ž+ $€=€Ȁ€€‚ÿL. Adleman. Algorithmic number theory - the complexity contribution. In Proc. 35th IEEE Symp. Foundations of Computer Science (FOCS), pages 88-113, 1994.2 –‘Ž' €€€€‚ÿAdl94b§}_Ž8* $€ú€Ȁ€€‚ÿL. M. Adleman. Molecular computations of solutions to combinatorial problems. Science, 266:1021-1024, November 11, 1994.0 ‘Žh' €€€€‚ÿAE83Š|8À* $€ø€Ȁ€€‚ÿD. Avis and H. ElGindy. A combinatorial approach to polygon similarity. IEEE Trans. Inform. Theory, IT-2:14hÀ…ƒ8-150, 1983.0 hJÀ' €€€€‚ÿAF92Ë ÀÁ+ $€A€Ȁ€€‚ÿD. Avis and K. Fukuda. A pivoting algorithm for convex hulls and vertex enumeration of arrangements and polyhedra. Discrete Comput. Geom., 8:295-313, 1992.0 JÀEÁ' €€€€‚ÿAG86°…ÁõÁ+ $€ €Ȁ€€‚ÿA. Apostolico and R. Giancarlo. The Boyer-Moore-Galil string searching strategies revisited. SIAM J. Computing, 15:98-105, 1986.2 EÁ'Â' €€€€‚ÿAGSS89â·õÁ Ã+ $€o€Ȁ€€‚ÿA. Aggarwal, L. Guibas, J. Saxe, and P. Shor. A linear-time algorithm for computing the Voronoi diagram of a convex polygon. Discrete and Computational Geometry, 4:591-604, 1989.1 'Â:Ã' €€€€‚ÿAGU72¡w ÃÛÃ* $€î€Ȁ€€‚ÿA. Aho, M. Garey, and J. Ullman. The transitive reduction of a directed graph. SIAM J. Computing, 1:131-137, 1972.1 :à Ä' €€€€‚ÿAho90íÂÛÃùÄ+ $€…€Ȁ€€‚ÿA. Aho. Algorithms for finding patterns in strings. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science: Algorithms and Complexity, volume A, pages 255-300. MIT Press, 1990.1 Ä*Å' €€€€‚ÿAHU74€zùÄÎÅ* $€ô€Ȁ€€‚ÿA. Aho, J. Hopcroft, and J. Ullman. The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading MA, 1974.1 *ÅÿÅ' €€€€‚ÿAHU83”jÎÅ“Æ* $€Ô€Ȁ€€‚ÿA. Aho, J. Hopcroft, and J. Ullman. Data Structures and Algorithms. Addison-Wesley, Reading MA, 1983.1 ÿÅÄÆ' €€€€‚ÿAig88d:“Æ(Ç* $€t€Ȁ€€‚ÿM. Aigner. Combinatorial Search. Wiley-Teubner, 1988.0 ÄÆXÇ' €€€€‚ÿAK89Ö«(Ç.È+ $€W€Ȁ€€‚ÿE. Aarts and J. Korst. Simulated annealing and Boltzman machines: A stochastic approach to combinatorial optimization and neural computing. John Wiley and Sons, 1989.1 XÇ_È' €€€€‚ÿAKD83º.ÈÉ+ $€€Ȁ€€‚ÿJ. H. Ahrens, K. D. Kohrt, and U. Dieter. Sampling from gamma and Poisson distributions. ACM Trans. Math. Softw., 9(2):255-257, June 1983.0 _ÈIÉ' €€€€‚ÿAM93Ï€ÉÊ+ $€I€Ȁ€€‚ÿS. Arya and D. Mount. Approximate nearest neighbor queries in fixed dimensions. In Proc. Fourth ACM-SIAM Symp. Discrete Algorithms (SODA), pages 271-280, 1993.= IÉUÊ0 0€€€€†"€‚ÿAMN 94òÇÊGË+ $€€Ȁ€€‚ÿS. Arya, D. Mount, N. Netanyahu, R. Silverman, and A. Wu. Approximate nearest neighbor queries in fixed dimensions. In Proc. Fifth ACM-SIAM Symp. Discrete Algorithms (SODA), pages 573-582, 1994.1 UÊxË' €€€€‚ÿAMO93ŒbGËÌ* $€Ä€Ȁ€€‚ÿR. Ahuja, T. Magnanti, and J. Orlin. Network Flows. Prentice Hall, Englewood Cliffs NJ, 1993.2 xË6Ì' €€€€‚ÿAMOT88ğÌúÌ% €?€Ȁ‚ÿR. Ahuja, K. Mehlhorn, J. Orlin, and R. Tarjan. Faster algorithms for the shortest path problem. Technical Report 193, MIT Operations Research Center, 1988.2 6Ì,Í' €€€€‚ÿAMWW88ØúÌïÍ+ $€1€Ȁ€€‚ÿH. Alt, K. Mehlhorn, H. Wagener, and E. Welzl. Congruence, similarity and symmetries of geometric objects. Discrete Comput. Geom., 3:237-256, 1988.1 ,Í Î' €€€€‚ÿAnd76zPïÍšÎ* $€ €Ȁ€€‚ÿG. Andrews. The Theory of Partitions. Addison-Wesley, Reading, Mass., 1976.1 ÎËÎ' €€€€‚ÿApo85ŚšÎÏ+ $€5€Ȁ€€‚ÿA. Apostolico. The myriad virtues of subword trees. In A. Apostolico and Z. Galil, editors, Combinatorial algorithms on words. Springer-Verlag, 1985.1 ËÎÁÏ' €€€€‚ÿAPT79Ð¥Ï+ $€K€Ȁ€€‚ÿB. Aspvall, M. PlasÁϝ…ƒs, and R. Tarjan. A linear-time algorithm for testing the truth of certain quantified boolean formulas. Info. Proc. Letters, 8:121-123, 1979.1 ÁÏÎ' €€€€‚ÿAro96ᶝ¯+ $€m€Ȁ€€‚ÿS. Arora. Polynomial time approximations schemes for Euclidean TSP and other geometric problems. In Proc. 37th IEEE Foundations of Computer Science (FOCS '96), pages 1-10, 1996.0 Îß' €€€€‚ÿAS89°…¯+ $€ €Ȁ€€‚ÿC. Aragon and R. Seidel. Randomized search trees. In Proc. 30th IEEE Symp. Foundations of Computer Science, pages 540-545, 1989.1 ßÀ' €€€€‚ÿAta83­‚m+ $€€Ȁ€€‚ÿM. Atallah. A linear time algorithm for the Hausdorff distance between convex polygons. Info. Proc. Letters, 8:207-209, 1983.1 Àž' €€€€‚ÿAta84–lm4* $€Ø€Ȁ€€‚ÿM. Atallah. Checking similarity of planar figures. Internat. J. Comput. Inform. Sci., 13:279-290, 1984.1 že' €€€€‚ÿAur91£y4* $€ò€Ȁ€€‚ÿF. Aurenhammer. Voronoi diagrams: a survey of a fundamental data structure. ACM Computing Surveys, 23:345-405, 1991.1 e9' €€€€‚ÿBaa88Už* $€ª€Ȁ€€‚ÿS. Baase. Computer Algorithms. Addison-Wesley, Reading MA, second edition, 1988.2 9ê' €€€€‚ÿBCGR92Ï€ž¹+ $€I€Ȁ€€‚ÿD. Berque, R. Cecchini, M. Goldberg, and R. Rivenburgh. The SetPlayer system for symbolic computation on power sets. J. Symbolic Computation, 14:645-662, 1992.1 êê' €€€€‚ÿBCW90c¹w* $€Æ€Ȁ€€‚ÿT. Bell, J. Cleary, and I. Witten. Text Compression. Prentice Hall, Englewood Cliffs NJ, 1990.1 êš' €€€€‚ÿBDH97žw`+ $€€Ȁ€€‚ÿC. Barber, D. Dobkin, and H. Huhdanpaa. The Quickhull algorithm for convex hulls. ACM Trans. on Mathematical Software, 22:469-483, 1997.1 š‘' €€€€‚ÿBel58‚X` * $€°€Ȁ€€‚ÿR. Bellman. On a routing problem. Quarterly of Applied Mathematics, 16:87-90, 1958.1 ‘D ' €€€€‚ÿBen75°… ô + $€ €Ȁ€€‚ÿJ. L. Bentley. Multidimensional binary search trees used for associative searching. Communications of the ACM, 18:509-517, 1975.1 D % ' €€€€‚ÿBen86pFô • * $€Œ€Ȁ€€‚ÿJ. Bentley. Programming Pearls. Addison-Wesley, Reading MA, 1986.1 % Æ ' €€€€‚ÿBen90uK• ; * $€–€Ȁ€€‚ÿJ. Bentley. More Programming Pearls. Addison-Wesley, Reading MA, 1990.2 Æ m ' €€€€‚ÿBen92a™o;  * $€Þ€Ȁ€€‚ÿJ. Bentley. Fast algorithms for geometric traveling salesman problems. ORSA J. Computing, 4:387-411, 1992.2 m 8 ' €€€€‚ÿBen92b–l Î * $€Ø€Ȁ€€‚ÿJ. L. Bentley. Software exploratorium: The trouble with qsort. UNIX Review, 10(2):85-93, February 1992.1 8 ÿ ' €€€€‚ÿBer89e;Î d * $€v€Ȁ€€‚ÿC. Berge. Hypergraphs. North-Holland, Amsterdam, 1989.2 ÿ – ' €€€€‚ÿBETT94Ù®d o+ $€]€Ȁ€€‚ÿG. Di Battista, P. Eades, R. Tamassia, and I. Tollis. Algorithms for drawing graphs: An annotated bibliography. Computational Geometry: Theory and Applications, 4, 1994.= – ¬0 0€€€€†"€‚ÿBFP 72³ˆo_+ $€€Ȁ€€‚ÿM. Blum, R. Floyd, V. Pratt, R. Rivest, and R. Tarjan. Time bounds for selection. J. Computer and System Sciences, 7:448-461, 1972.0 ¬' €€€€‚ÿBG95íÂ_ˆ@+ $€…€Ȁ€€‚ÿJ. Berry and M. Goldberg. Path optimization and near-greedy analysis ˆ@…ƒfor graph partitioning: An empirical study. In Proc. 6th ACM-SIAM Symposium on Discrete Algorithms, pages 223-232, 1995.= Å@0 0€€€€†"€‚ÿBGL 95܈@ÌA+ $€¹€Ȁ€€‚ÿC. Di Battista, A. Garg, G. Liotta, R. Tamassia, E. Tassinari, and F. Vargiu. An experimental comparison of three graph drawing algorithms. In Proc. 11th ACM Symposium on Computational Geometry, pages 306-315, 1995.1 Å@ýA' €€€€‚ÿBGS95çŒÌAäB+ $€y€Ȁ€€‚ÿM Bellare, O. Goldreich, and M. Sudan. Free bits, PCPs, and non-approximability - towards tight results. In Proc. IEEE 36th Symp. Foundations of Computer Science, pages 422-431, 1995.0 ýAC' €€€€‚ÿBH90‰_äBC* $€Ÿ€Ȁ€€‚ÿF. Buckley and F. Harary. Distances in Graphs. Addison-Wesley, Redwood City, Calif., 1990.0 CÍC' €€€€‚ÿBJ85œŒCŠD1 0€€Ȁ€€€€‚ÿS. Bent and J. John. Finding the median requires 2n comparisons. In Proc. 17th ACM Symp. on Theory of Computing, pages 213-216, 1985.= ÍCÇD0 0€€€€†"€‚ÿBJL 91Ú¯ŠD¡E+ $€_€Ȁ€€‚ÿA. Blum, T. Jiang, M. Li, J. Tromp, and M. Yanakakis. Linear approximation of shortest superstrings. In Proc. 23rd ACM Symp. on Theory of Computing., pages 328-336, 1991.2 ÇDÓE' €€€€‚ÿBJLM83乡E·F+ $€s€Ȁ€€‚ÿJ. Bentley, D. Johnson, F. Leighton, and C. McGeoch. An experimental study of bin packing. In Proc. 21st Allerton Conf. on Communication, Control, and Computing, pages 51-60, 1983.0 ÓEçF' €€€€‚ÿBL77¶‹·FG+ $€€Ȁ€€‚ÿB. P. Buckles and M. Lybanon. Generation of a vector from the lexicographical index. ACM Trans. Math. Softw., 3(2):180-182, June 1977.1 çFÎG' €€€€‚ÿBLS91œ’G‹H+ $€%€Ȁ€€‚ÿD. Bailey, K. Lee, and H. Simon. Using Strassen's algorithm to accelerate the solution of linear systems. J. Supercomputing, 4:357-371, 1991.1 ÎGŒH' €€€€‚ÿBlu67ãž‹HŸI+ $€q€Ȁ€€‚ÿH. Blum. A transformation for extracting new descriptions of shape. In W. Wathen-Dunn, editor, Models for the Perception of speech and Visual Form, pages 362-380. MIT Press, 1967.1 ŒHÐI' €€€€‚ÿBLW76‘gŸIaJ* $€Î€Ȁ€€‚ÿN. L. Biggs, E. K. Lloyd, and R. J. Wilson. Graph Theory 1736-1936. Clarendon Press, Oxford, 1976.0 ÐI‘J' €€€€‚ÿBM53‚XaJK* $€°€Ȁ€€‚ÿG. Birkhoff and S. MacLane. A survey of modern algebra. Macmillian, New York, 1953.0 ‘JCK' €€€€‚ÿBM72£yKæK* $€ò€Ȁ€€‚ÿR. Bayer and E. McCreight. Organization and maintenance of large ordered indexes. Acta Informatica, 1:173-189, 1972.0 CKL' €€€€‚ÿBM77•kæK«L* $€Ö€Ȁ€€‚ÿR. Boyer and J. Moore. A fast string-searching algorithm. Communications of the ACM, 20:762-772, 1977.0 LÛL' €€€€‚ÿBM89€z«LM* $€ô€Ȁ€€‚ÿJ. Boreddy and R. N. Mukherjee. An algorithm to find polygon similarity. Inform. Process. Lett., 33(4):205-206, 1989.0 ÛL¯M' €€€€‚ÿBO79œ’MlN+ $€%€Ȁ€€‚ÿJ. Bentley and T. Ottmann. Algorithms for reporting and counting geometric intersections. IEEE Transactions on Computers, C-28:643-647, 1979.0 ¯MœN' €€€€‚ÿBO83°…lNLO+ $€ €Ȁ€€‚ÿM. Ben-Or. Lower bounds for algebraic computation trees. In Proc. Fifteenth ACM Symp. on Theory of Computing, pages 80-86, 1983.0 œN|O' €€€€‚ÿBP76ƒYLO €* $€²€Ȁ€€‚ÿE. Balas and M. Padberg. Set partitioning - a survey. SIAM Review, 18:710-760, 1976.|O €…ƒ0 |O<€' €€€€‚ÿBR80ɕ €4 6€-€Ȁ†"€€€‚ÿI. Barrodale and F. D. K. Roberts. Solution of the constrained linear approximation problem. ACM Trans. Math. Softw., 6(2):231-235, June 1980.0 <€5' €€€€‚ÿBR95ŽdÁ* $€È€Ȁ€€‚ÿA. Binstock and J. Rex. Practical Algorithms for Programmers. Addison-Wesley, Reading MA, 1995.1 5ô' €€€€‚ÿBre73“iÁ‡‚* $€Ò€Ȁ€€‚ÿR. Brent. Algorithms for minimization without derivatives. Prentice-Hall, Englewood Cliffs NJ, 1973.1 ôž‚' €€€€‚ÿBre74’h‡‚Jƒ* $€Ð€Ȁ€€‚ÿR. P. Brent. A Gaussian pseudo-random number generator. Commun. ACM, 17(12):704-706, December 1974.1 ž‚{ƒ' €€€€‚ÿBrè79”jJƒ„* $€Ô€Ȁ€€‚ÿD. Brèlaz. New methods to color the vertices of a graph. Communications of the ACM, 22:251-256, 1979.1 {ƒ@„' €€€€‚ÿBri74€V„À„* $€¬€Ȁ€€‚ÿE. Brigham. The Fast Fourier Transform. Prentice Hall, Englewood Cliffs NJ, 1974.1 @„ñ„' €€€€‚ÿBro74sIÀ„d…* $€’€Ȁ€€‚ÿF. Brooks. The Mythical Man-Month. Addison-Wesley, Reading MA, 1974.1 ñ„•…' €€€€‚ÿBro88‰_d…†* $€Ÿ€Ȁ€€‚ÿR. Brown. Calendar queuing method for future event list manipulation. Comm. ACM, 30, 1988.1 •…O†' €€€€‚ÿBrz64{Q†ʆ* $€¢€Ȁ€€‚ÿJ. Brzozowski. Derivatives of regular expressions. J. ACM, 11:481-494, 1964.0 O†ú†' €€€€‚ÿBS76œ’ʆ·‡+ $€%€Ȁ€€‚ÿJ. Bentley and M. Shamos. Divide-and-conquer in higher-dimensional space. In Proc. Eighth ACM Symp. Theory of Computing, pages 220-230, 1976.0 ú†ç‡' €€€€‚ÿBS81¯|·‡–ˆ3 6€ú€Ȁ†"€€€‚ÿI. Barrodale and G. F. Stuart. A Fortran program for solving . ACM Trans. Math. Softw., 7(3):391-397, September 1981.0 ç‡ƈ' €€€€‚ÿBS86©–ˆo‰* $€þ€Ȁ€€‚ÿG. Berry and R. Sethi. From regular expressions to deterministic automata. Theoretical Computer Science, 48:117-126, 1986.0 ƈŸ‰' €€€€‚ÿBS93¡wo‰@Š* $€î€Ȁ€€‚ÿE. Biham and A. Shamir. Differential Cryptanalysis of the Data Encryption Standard. Springer-Verlag, Berlin, 1993.0 Ÿ‰pŠ' €€€€‚ÿBS96 v@Š‹* $€ì€Ȁ€€‚ÿE. Bach and J. Shallit. Algorithmic Number Theory: Efficient Algorithms, volume 1. MIT Press, Cambridge MA, 1996.0 pŠ@‹' €€€€‚ÿBS97Ø‹Œ+ $€1€Ȁ€€‚ÿR. Bradley and S. Skiena. Fabricating arrays of strings. In Proc. First Int. Conf. Computational Molecular Biology (RECOMB '97), pages 57-66, 1997.0 @‹3Œ' €€€€‚ÿBT92†\Œ¹Œ* $€ž€Ȁ€€‚ÿJ. Buchanan and P. Turner. Numerical methods and analysis. McGraw-Hill, New York, 1992.1 3ŒêŒ' €€€€‚ÿBuc94²‡¹Œœ+ $€€Ȁ€€‚ÿA. G. Buckley. A Fortran 90 code for unconstrained nonlinear minimization. ACM Trans. Math. Softw., 20(3):354-372, September 1994.0 êŒ̍' €€€€‚ÿBW91Ÿ“œŠŽ+ $€'€Ȁ€€‚ÿG. Brightwell and P. Winkler. Counting linear extensions is #P-complete. In Proc. 23rd ACM Symp. Theory Computing (STOC), pages 175-181, 1991.3 ̍œŽ' €€€€‚ÿBYGNR94)þŠŽ À+ $€ý€Ȁ€€‚ÿR. Bar-Yehuda, D. Geiger, J. Naor, and R. Roth. Approximation algorithms for the vertex feedback set problem with applications to constraint satisfaction and Bayesian inference. In Proc. Fifth ACM-SIAM Symp. Discrete Algorithms, pages 344-354, 1994.œŽ À…ƒ1 œŽ=À' €€€€‚ÿCan87€V ÀœÀ* $€¬€Ȁ€€‚ÿJ. Canny. The complexity of robot motion planning. MIT Press, Cambridge MA, 1987.0 =ÀíÀ' €€€€‚ÿCC92¬œÀ™Á+ $€€Ȁ€€‚ÿS. Carlsson and J. Chen. The complexity of heaps. In Proc. Third ACM-SIAM Symp. on Discrete Algorithms, pages 393-402, 1992.2 íÀËÁ' €€€€‚ÿCCDG82ؙÁŽÂ+ $€1€Ȁ€€‚ÿP. Chinn, J. Chvátolvá, A. K. Dewdney, and N. E. Gibbs. The bandwidth problem for graphs and matrices - a survey. J. Graph Theory, 6:223-254, 1982.0 ËÁŸÂ' €€€€‚ÿCD85ȝŽÂ†Ã+ $€;€Ȁ€€‚ÿB. Chazelle and D. Dobkin. Optimal convex decompositions. In G. Toussaint, editor, Computational Geometry, pages 63-133. North-Holland, Amsterdam, 1985.1 ŸÂ·Ã' €€€€‚ÿCDT95ðņçÄ+ $€‹€Ȁ€€‚ÿG. Carpento, M. Dell'Amico, and P. Toth. CDT: A subroutine for the exact solution of large-scale, asymmetric traveling salesman problems. ACM Trans. Math. Softw., 21(4):410-415, December 1995.0 ·Ã×Ä' €€€€‚ÿCE92›q§ÄrÅ* $€â€Ȁ€€‚ÿB. Chazelle and H. Edelsbrunner. An optimal algorithm for intersecting line segments. J. ACM, 39:1-54, 1992.0 ×Ä¢Å' €€€€‚ÿCG94ÝžrÅÆ% €q€Ȁ‚ÿB. Cherkassky and A. Goldberg. On implementing push-relabel method for the maximum flow problem. Technical Report 94-1523, Department of Computer Science, Stanford University, 1994.1 ¢Å°Æ' €€€€‚ÿCGJ96گƊÇ+ $€_€Ȁ€€‚ÿE. G. Coffman, M. R. Garey, and D. S. Johnson. Approximation algorithms for bin packing: a survey. In D. Hochbaum, editor, Approximation algorithms. PWS Publishing, 1996.1 °Æ»Ç' €€€€‚ÿCGL85Š`ŠÇEÈ* $€À€Ȁ€€‚ÿB. Chazelle, L. Guibas, and D. T. Lee. The power of geometric duality. BIT, 25:76-90, 1985.2 »ÇwÈ' €€€€‚ÿCGPS76Ò§EÈIÉ+ $€O€Ȁ€€‚ÿH. L. Crane Jr., N. F. Gibbs, W. G. Poole Jr., and P. K. Stockmeyer. Matrix bandwidth and profile reduction. ACM Trans. Math. Softw., 2(4):375-377, December 1976.1 wÈzÉ' €€€€‚ÿCGR93æÁIÉ`Ê% €ƒ€Ȁ‚ÿB. Cherkassky, A. Goldberg, and T. Radzik. Shortest paths algorithms: theory and experimental evaluation. Technical Report 93-1480, Department of Computer Science, Stanford University, 1993.1 zÉ‘Ê' €€€€‚ÿCha71uK`ÊË* $€–€Ȁ€€‚ÿJ. M. Chambers. Partial sorting. Commun. ACM, 14(5):357-358, May 1971.1 ‘Ê7Ë' €€€€‚ÿCha91 vË×Ë* $€ì€Ȁ€€‚ÿB. Chazelle. Triangulating a simple polygon in linear time. Discrete and Computational Geometry, 6:485-524, 1991.1 7ËÌ' €€€€‚ÿChe80Û°×ËãÌ+ $€a€Ȁ€€‚ÿT-Y. Cheung. A program for the multifacility location problem with rectilinear distance by the minimum-cut approach. ACM Trans. Math. Softw., 6(3):430-431, September 1980.1 ÌÍ' €€€€‚ÿChr76ôÏãÌÎ% €Ÿ€Ȁ‚ÿN. Christofides. Worst-case analysis of a new heuristic for the traveling salesman problem. Technical report, Graduate School of Industrial Administration, Carnegie-Mellon University, Pittsburgh PA, 1976.1 Í9Î' €€€€‚ÿCHT90Ÿ{ÎØÎ$ €ö€Ȁ‚ÿJ. Cai, X. Han, and R. Tarjan. New solutions to four planar graph problems. Technical report, New York University, 1990.1 9Î Ï' €€€€‚ÿChv83lBØÎuÏ* $€„€Ȁ€€‚ÿV. Chvatal. Linear Programming. Freeman, San Francisco, 1983.0 Ï¥Ï' €€€€‚ÿCK70€VuÏ1* $€¬€Ȁ€€‚ÿD. Chand and S. Kapur. An algorithm for convex p¥Ï1…ƒolytopes. J. ACM, 17:78-86, 1970.0 ¥Ïa' €€€€‚ÿCK75Ž‰1+ $€€Ȁ€€‚ÿN. Christofides and S. Korman. A computational survey of methods for the set covering problem. Management Science, 21:591-599, 1975.0 aE' €€€€‚ÿCK80cÒ* $€Æ€Ȁ€€‚ÿW. Cheney and D. Kincaid. Numerical Mathematics and Computing. Brooks/Cole, Monterey CA, 1980.0 E' €€€€‚ÿCK94Š|Òš* $€ø€Ȁ€€‚ÿA. Chetverin and F. Kramer. Oligonucleotide arrays: New concepts and possibilities. Bio/Technology, 12:1093-1099, 1994.1 Ù' €€€€‚ÿCla92Ð¥š©+ $€K€Ȁ€€‚ÿK. L. Clarkson. Safe and effective determinant evaluation. In Proc. 31st IEEE Symposium on Foundations of Computer Science, pages 387-395, Pittsburgh, PA, 1992.1 ÙÚ' €€€€‚ÿCLR90‘g©k* $€Î€Ȁ€€‚ÿT. Cormen, C. Leiserson, and R. Rivest. Introduction to Algorithms. MIT Press, Cambridge MA, 1990.0 Ú›' €€€€‚ÿCM69°…kK+ $€ €Ȁ€€‚ÿE. Cuthill and J. McKee. Reducing the bandwidth of sparse symmetric matrices. In Proc. 24th Nat. Conf. ACM, pages 157-172, 1969.1 ›|' €€€€‚ÿCof76sIKï* $€’€Ȁ€€‚ÿE. Coffman. Computer and Job shop Scheduling. Wiley, New York, 1976.1 | ' €€€€‚ÿCoh94Ï€ïï+ $€I€Ȁ€€‚ÿE. Cohen. Estimating the size of the transitive closure in linear time. In 35th Annual Symposium on Foundations of Computer Science, pages 190-200. IEEE, 1994.1  ' €€€€‚ÿCoo71š~ïÈ* $€ü€Ȁ€€‚ÿS. Cook. The complexity of theorem proving procedures. In Proc. Third ACM Symp. Theory of Computing, pages 151-158, 1971.0 ø' €€€€‚ÿCP90À•Èž+ $€+€Ȁ€€‚ÿR. Carraghan and P. Paradalos. An exact algorithm for the maximum clique problem. In Operations Research Letters, volume 9, pages 375-382, 1990.0 øè' €€€€‚ÿCR76«€ž“ + $€€Ȁ€€‚ÿJ. Cohen and M. Roth. On the implementation of Strassen's fast multiplication algorithm. Acta Informatica, 6:341-355, 1976.0 èà ' €€€€‚ÿCR94…[“ H * $€¶€Ȁ€€‚ÿM. Crochemore and W. Rytter. Text Algorithms. Oxford University Press, New York, 1994.1 à y ' €€€€‚ÿCra94„ZH ý * $€Ž€Ȁ€€‚ÿR. Crandall. Topics in Scientific Computation. Telos/Springer-Verlag, New York, 1994.0 y - ' €€€€‚ÿCS93eý Œ * $€Ê€Ȁ€€‚ÿJ. Conway and N. Sloane. Sphere packings, lattices, and groups. Springer-Verlag, New York, 1993.0 - ì ' €€€€‚ÿCT65¹ŽŒ ¥ + $€€Ȁ€€‚ÿJ. Cooley and J. Tukey. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 19:297-301, 1965.0 ì Õ ' €€€€‚ÿCT71’h¥ g * $€Ð€Ȁ€€‚ÿM. W. Coleman and M. S. Taylor. Circular integer partitioning. Commun. ACM, 14(1):48, January 1971.0 Õ — ' €€€€‚ÿCT80Ÿug 6* $€ê€Ȁ€€‚ÿG. Carpaneto and P. Toth. Solution of the assignment problem. ACM Trans. Math. Softw., 6(1):104-111, March 1980.0 — f' €€€€‚ÿCT92—m6ý* $€Ú€Ȁ€€‚ÿY. Chiang and R. Tamassia. Dynamic algorithms in computational geometry. Proc. IEEE, 80:1412-1434, 1992.0 f-' €€€€‚ÿCW79ÝýA@+ $€»€Ȁ€€‚ÿB. Commentz-Walter. A string matching algorithm fast on the average. In Proc. Sixth Int. Coll. on Automata, Languages, and Programming (ICALP), pages 118-132. Spring-A@…ƒer Verlag, Lecture Notes in Computer Science, 1979.0 -q@' €€€€‚ÿCW87ƛA@7A+ $€7€Ȁ€€‚ÿD. Coppersmith and S. Winograd. Matrix multiplication via arithmetic progressions. In Proc. Nineteenth ACM Symp. Theory of Computing, pages 1-6, 1987.1 q@hA' €€€€‚ÿDan63c7AõA* $€Æ€Ȁ€€‚ÿG. Dantzig. Linear programming and extensions. Princeton University Press, Princeton NJ, 1963.1 hA&B' €€€€‚ÿDau92qGõA—B* $€Ž€Ȁ€€‚ÿI. Daubechies. Ten Lectures on Wavelets. SIAM, Philadelphia, 1992.0 &BÇB' €€€€‚ÿDB74‡]—BNC* $€º€Ȁ€€‚ÿG. Dahlquist and A. Bjorck. Numerical Methods. Prentice-Hall, Englewood Cliffs NJ, 1974.0 ÇB~C' €€€€‚ÿDB86ŸuNCD* $€ê€Ȁ€€‚ÿG. Davies and S. Bowsher. Algorithms for pattern matching. Software - Practice and Experience, 16:575-601, 1986.1 ~CND' €€€€‚ÿDen82|RDÊD* $€€€Ȁ€€‚ÿD. Denning. Cryptography and Data Security. Addison-Wesley, Reading MA, 1982.0 NDúD' €€€€‚ÿDF79§}ÊD¡E* $€ú€Ȁ€€‚ÿE. Denardo and B. Fox. Shortest-route methods: 1. reaching, pruning, and buckets. Operations Research, 27:161-186, 1979.1 úDÒE' €€€€‚ÿDFJ54µŠ¡E‡F+ $€€Ȁ€€‚ÿG. Dantzig, D. Fulkerson, and S. Johnson. Solution of a large-scale traveling-salesman problem. Operations Research, 2:393-410, 1954.2 ÒE¹F' €€€€‚ÿdFPP88滇FŸG+ $€w€Ȁ€€‚ÿH. de Fraysseix, J. Pach, and R. Pollack. Small sets supporting Fary embeddings of planar graphs. In Proc. of the 20th Symposium on the Theory of Computing, pages 426-433. ACM, 1988.0 ¹FÏG' €€€€‚ÿDG87²‡ŸGH+ $€€Ȁ€€‚ÿJ. Dongarra and E. Grosse. Distribution of mathematical software via electronic mail. Communications of the ACM, 30:403-407, 1987.2 ÏG³H' €€€€‚ÿDGKK79滁H™I+ $€w€Ȁ€€‚ÿR. Dial, F. Glover, D. Karney, and D. Klingman. A computational analysis of alternative algorithms and labeling techniques for finding shortest path trees. Networks, 9:215-248, 1979.0 ³HÉI' €€€€‚ÿDH73’h™I[J* $€Ð€Ȁ€€‚ÿR. Duda and P. Hart. Pattern Classification and Scene Analysis. Wiley-Interscience, New York, 1973.0 ÉI‹J' €€€€‚ÿDH92¢x[J-K* $€ð€Ȁ€€‚ÿD. Du and F. Hwang. A proof of Gilbert and Pollak's conjecture on the Steiner ratio. Algorithmica, 7:121-135, 1992.1 ‹J^K' €€€€‚ÿDij59—m-KõK* $€Ú€Ȁ€€‚ÿE. W. Dijkstra. A note on two problems in connection with graphs. Nuerische Mathematik, 1:269-271, 1959.0 ^K%L' €€€€‚ÿDJ92›qõKÀL* $€â€Ȁ€€‚ÿG. Das and D. Joseph. Minimum vertex hulls for polyhedral domains. Theoret. Comput. Sci., 103:107-135, 1992.0 %LðL' €€€€‚ÿDL76fÀL€M* $€Ì€Ȁ€€‚ÿD. Dobkin and R. Lipton. Multidimensional searching problems. SIAM J. Computing, 5:181-186, 1976.1 ðL±M' €€€€‚ÿDLR79§}€MXN* $€ú€Ȁ€€‚ÿD. Dobkin, R. Lipton, and S. Reiss. Linear programming is log-space hard for P. Info. Processing Letters, 8:96-97, 1979.0 ±MˆN' €€€€‚ÿDM80eXNO* $€Ê€Ȁ€€‚ÿD. Dobkin and J. I. Munro. Determining the mode. Theoretical Computer Science, 12:255-263, 1980.0 ˆNGO' €€€€‚ÿDM97š~O €* $€ü€Ȁ€€‚ÿK. Daniels and V. Milenkovic. Multiple translational containment. part I: an approximation algorithm. Algorithmica, 1997.GO €…ƒ2 GO>€' €€€€‚ÿDMBS79œr €ڀ* $€ä€Ȁ€€‚ÿJ. Dongarra, C. Moler, J. Bunch, and G. Stewart. LINPACK User's Guide. SIAM Publications, Philadelphia, 1979.2 >€ ' €€€€‚ÿDNRT81۰ڀç+ $€a€Ȁ€€‚ÿJ. J. Ducroz, S. M. Nugent, J. K. Reid, and D. B. Taylor. Solution of real linear equations in a paged virtual store. ACM Trans. Math. Softw., 7(4):537-541, December 1981.1 ‚' €€€€‚ÿDor94µŠç͂+ $€€Ȁ€€‚ÿS. Dorward. A survey of object-space hidden surface removal. Int. J. Computational Geometry Theory and Applications, 4:325-362, 1994.0 ‚ý‚' €€€€‚ÿDP73ôÉ͂ñƒ+ $€“€Ȁ€€‚ÿD. H. Douglas and T. K. Peucker. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Canadian Cartographer, 10(2):112-122, December 1973.0 ý‚!„' €€€€‚ÿDP84‘gñƒ²„* $€Î€Ȁ€€‚ÿN. Deo and C. Pang. Shortest path algorithms: Taxonomy and annotation. Networks, 14:275-323, 1984.0 !„â„' €€€€‚ÿDR90s²„…* $€æ€Ȁ€€‚ÿN. Dershowitz and E. Reingold. Calendrical calculations. Software - Practice and Experience, 20:899-928, 1990.0 ℯ…' €€€€‚ÿDR97“i…B†* $€Ò€Ȁ€€‚ÿN. Dershowitz and E. Reingold. Calendrical Calculations. Cambridge University Press, New York, 1997.= ¯…†0 0€€€€†"€‚ÿDRR 951B†°‡+ $€ €Ȁ€€‚ÿS. Dawson, C.R. Ramakrishnan, I.V. Ramakrishnan, K. Sagonas, S. Skiena, T. Swift, and D.S. Warren. Unification factoring for efficient execution of logic programs. In 22nd ACM Symposium on Principles of Programming Languages (POPL '95), pages 247-258, 1995.0 †à‡' €€€€‚ÿDS88ž°‡˜ˆ+ $€€Ȁ€€‚ÿD. Dobkin and D. Silver. Recipes for geometry and numerical analysis. In Proc. 4th ACM Symp. Computational Geometry, pages 93-105, 1988.1 à‡Ɉ' €€€€‚ÿDuf81•k˜ˆ^‰* $€Ö€Ȁ€€‚ÿS. Duff. Permutations for a zero-free diagonal. ACM Trans. Math. Softw., 7(3):387-390, September 1981.1 Ɉ‰' €€€€‚ÿdVS82£y^‰2Š* $€ò€Ȁ€€‚ÿG. de V. Smit. A comparison of three string matching algorithms. Software - Practice and Experience, 12:57-66, 1982.0 ‰bŠ' €€€€‚ÿDZ95Š|2Š‹* $€ø€Ȁ€€‚ÿD. Dor and U. Zwick. Selecting the median. In Proc. Sixth ACM-SIAM Symp. Discrete Algorithms (SODA), pages 28-37, 1995.1 bŠ9‹' €€€€‚ÿEbe88xN‹±‹* $€œ€Ȁ€€‚ÿJ. Ebert. Computing Eulerian trails. Info. Proc. Letters, 28:93-97, 1988.1 9‹â‹' €€€€‚ÿEdd77¥{±‹‡Œ* $€ö€Ȁ€€‚ÿW. F. Eddy. CONVEX: a new convex hull algorithm for planar sets. ACM Trans. Math. Softw., 3(4):411-412, December 1977.1 â‹žŒ' €€€€‚ÿEde87…[‡Œ=* $€¶€Ȁ€€‚ÿH. Edelsbrunner. Algorithms for Combinatorial Geometry. Springer-Verlag, Berlin, 1987.1 žŒn' €€€€‚ÿEdm65zP=è* $€ €Ȁ€€‚ÿJ. Edmonds. Paths, trees, and flowers. Canadian J. Math., 17:449-467, 1965.1 nŽ' €€€€‚ÿEdm71ˆ^荡Ž* $€Œ€Ȁ€€‚ÿJ. Edmonds. Matroids and the greedy algorithm. Mathematical Programming, 1:126-136, 1971.0 Žю' €€€€‚ÿEG60œr¡Žm* $€ä€Ȁ€€‚ÿP. Erdös and T. Gallai. Graphs with prescribed degrees of vertices. Mat. Lapok (Hungarian), 11:264-274, 1960.0 ю' €€€€‚ÿEG89§}mPÀ* $€ú€Ȁ€€‚ÿH. Edelsbrunner and L. Guibas. Topologically sweeping anPÀ…ƒ arrangement. J. Computer and System Sciences, 38:165-194, 1989.0 €À' €€€€‚ÿEG91µŠPÀ5Á+ $€€Ȁ€€‚ÿH. Edelsbrunner and L. Guibas. Corrigendum: Topologically sweeping an arrangement. J. Computer and System Sciences, 42:249-251, 1991.2 €ÀgÁ' €€€€‚ÿEGIN92Ü5ÁnÂ+ $€¹€Ȁ€€‚ÿD. Eppstein, Z. Galil, G. F. Italiano, and A. Nissenzweig. Sparsification: A technique for speeding up dynamic graph algorithms. In Proc. 33rd IEEE Symp. on Foundations of Computer Science (FOCS), pages 60-69, 1992.1 gÁŸÂ' €€€€‚ÿEGS86±†nÂPÃ+ $€ €Ȁ€€‚ÿH. Edelsbrunner, L. Guibas, and J. Stolfi. Optimal point location in a monotone subdivision. SIAM J. Computing, 15:317-340, 1986.0 ŸÂ€Ã' €€€€‚ÿEJ73œrPÃÄ* $€ä€Ȁ€€‚ÿJ. Edmonds and E. Johnson. Matching, Euler tours, and the Chinese postman. Math. Programming, 5:88-124, 1973.1 €ÃMÄ' €€€€‚ÿEKA84ߎÄ,Å+ $€i€Ȁ€€‚ÿM. I. Edahiro, I. Kokubo, and T. Asano. A new point location algorithm and its practical efficiency - comparison with existing algorithms. ACM Trans. Graphics, 3:86-109, 1984.1 MÄ]Å' €€€€‚ÿEKS83Ś,Å"Æ+ $€5€Ȁ€€‚ÿH. Edelsbrunner, D. Kirkpatrick, and R. Seidel. On the shape of a set of points in the plane. IEEE Trans. on Information Theory, IT-29:551-559, 1983.1 ]ÅSÆ' €€€€‚ÿELS93º"Æ Ç+ $€€Ȁ€€‚ÿP. Eades, X. Lin, and W. F. Smyth. A fast and effective heuristic for the feedback arc set problem. Info. Proc. Letters, 47:319-323, 1993.0 SÆ=Ç' €€€€‚ÿEM94 v ÇÝÇ* $€ì€Ȁ€€‚ÿH. Edelsbrunner and Ernst P. Mücke. Three-dimensional alpha shapes. ACM Transactions on Graphics, 13:43-72, 1994.0 =Ç È' €€€€‚ÿES74eÝÇœÈ* $€Ê€Ȁ€€‚ÿP. Erdös and J. Spencer. Probabilistic Methods in Combinatorics. Academic Press, New York, 1974.0 ÈÌÈ' €€€€‚ÿES86€zœÈpÉ* $€ô€Ȁ€€‚ÿH. Edelsbrunner and R. Seidel. Voronoi diagrams and arrangements. Discrete and Computational Geometry, 1:25-44, 1986.1 ÌÈ¡É' €€€€‚ÿESS93°…pÉQÊ+ $€ €Ȁ€€‚ÿH. Edelsbrunner, R. Seidel, and M. Sharir. On the zone theorem for hyperplane arrangements. SIAM J. Computing, 22:418-429, 1993.1 ¡É‚Ê' €€€€‚ÿESV96ºQÊ<Ë+ $€€Ȁ€€‚ÿF. Evans, S. Skiena, and A. Varshney. Optimizing triangle strips for fast rendering. In Proc. IEEE Visualization '96, pages 319-326, 1996.1 ‚ÊmË' €€€€‚ÿEul36±†<ËÌ+ $€ €Ȁ€€‚ÿL. Euler. Solutio problematis ad geometriam situs pertinentis. Commentarii Academiae Scientiarum Petropolitanae, 8:128-140, 1736.2 mËPÌ' €€€€‚ÿEve79auKÌÅÌ* $€–€Ȁ€€‚ÿS. Even. Graph Algorithms. Computer Science Press, Rockville MD, 1979.2 PÌ÷Ì' €€€€‚ÿEve79bÕªÅÌÌÍ+ $€U€Ȁ€€‚ÿG. Everstine. A comparison of three resequencing algorithms for the reduction of matrix profile and wave-front. Int. J. Numerical Methods in Engr., 14:837-863, 1979./÷ÌûÍ' €€€€‚ÿF48”jÌ͏Î* $€Ô€Ȁ€€‚ÿI. Fáry. On straight line representation of planar graphs. Acta. Sci. Math. Szeged, 11:229-233, 1948.1 ûÍÀÎ' €€€€‚ÿFaj87WÎAÏ* $€®€Ȁ€€‚ÿS. Fajtlowicz. On conjectures of Graffiti. Discrete Mathematics, 72:113-118, 1987.0 ÀÎqÏ' €€€€‚ÿFF62ŽdAÏ * $€È€Ȁ€€‚ÿL. Ford and D. R. Fulkerson. Flows in Networks. Princeton University Press, Princeton NJ, 1962.qÏ …ƒ1 qÏ=' €€€€‚ÿFis95Š| ã* $€ø€Ȁ€€‚ÿP. Fishwick. Simulation Model Design and Execution: Building Digital Worlds. Prentice Hall, Englewood Cliffs, NJ, 1995.0 =' €€€€‚ÿFJ95¥{ãž* $€ö€Ȁ€€‚ÿA. Frieze and M. Jerrum. An analysis of a Monte Carlo algorithm for estimating the permanent. Combinatorica, 15, 1995.1 é' €€€€‚ÿFle74¡wžŠ* $€î€Ȁ€€‚ÿH. Fleischner. The square of every two-connected graph is Hamiltonian. J. Combinatorial Theory, B, 16:29-34, 1974.1 é»' €€€€‚ÿFle80¢xŠ]* $€ð€Ȁ€€‚ÿR. Fletcher. Practical Methods of Optimization: Unconstrained Optimization, volume 1. John Wiley, Chichester, 1980.1 »Ž' €€€€‚ÿFlo62~T] * $€š€Ȁ€€‚ÿR. Floyd. Algorithm 97 (shortest path). Communications of the ACM, 7:345, 1962.1 Ž=' €€€€‚ÿFlo64{Q ž* $€¢€Ȁ€€‚ÿR. Floyd. Algorithm 245 (treesort). Communications of the ACM, 18:701, 1964.0 =è' €€€€‚ÿFM71ȝž°+ $€;€Ȁ€€‚ÿM. Fischer and A. Meyer. Boolean matrix multiplication and transitive closure. In IEEE 12th Symp. on Switching and Automata Theory, pages 129-131, 1971.1 èá' €€€€‚ÿFor87…[°f* $€¶€Ȁ€€‚ÿS. Fortune. A sweepline algorithm for Voronoi diagrams. Algorithmica, 2:153-174, 1987.1 á—' €€€€‚ÿFor92Þ³fu+ $€g€Ȁ€€‚ÿS. Fortune. Voronoi diagrams and Delaunay triangulations. In D.-Z. Du and F. Hwang, editors, Computing in Euclidean Geometry, volume 1, pages 193-234. World Scientific, 1992.1 —Š' €€€€‚ÿFP75a¬uR+ $€€Ȁ€€‚ÿD. Fayard and G. Plateau. Resolution of the 0-1 knapsack problem: Comparison of methods. Math. Programming, 8:272-307, 1975.1 Šƒ' €€€€‚ÿFP75båºRh + $€u€Ȁ€€‚ÿH. Feng and T. Pavlidis. Decomposition of polygons into simpler components: feature generation for syntactic pattern recognition. IEEE Transactions on Computers, C-24:636-650, 1975.0 ƒ˜ ' €€€€‚ÿFR75—mh / * $€Ú€Ȁ€€‚ÿR. Floyd and R. Rivest. Expected time bounds for selection. Communications of the ACM, 18:165-172, 1975.0 ˜ _ ' €€€€‚ÿFR94¶‹/  + $€€Ȁ€€‚ÿM. Fürer and B. Raghavachari. Approximating the minimum-degree Steiner tree to within one of optimal. J. Algorithms, 17:409-423, 1994.1 _ F ' €€€€‚ÿFra79Žd Ô * $€È€Ȁ€€‚ÿD. Fraser. An optimized mass storage FFT. ACM Trans. Math. Softw., 5(4):500-517, December 1979.1 F  ' €€€€‚ÿFre62sIÔ x * $€’€Ȁ€€‚ÿE. Fredkin. Trie memory. Communications of the ACM, 3:490-499, 1962.0  š ' €€€€‚ÿFT87¯„x W + $€ €Ȁ€€‚ÿM. Fredman and R. Tarjan. Fibonacci heaps and their uses in improved network optimization algorithms. J. ACM, 34:596-615, 1987.1 š ˆ ' €€€€‚ÿFuj96­‚W 5+ $€€Ȁ€€‚ÿT. Fujito. A note on approximation of the vertex cover and feedback vertex set problems. Info. Proc. Letters, 59:59-63, 1996.1 ˆ f' €€€€‚ÿFvW93Ś5++ $€5€Ȁ€€‚ÿS. Fortune and C. van Wyk. Efficient exact arithmetic for computational geometry. In Proc. 9th ACM Symp. Computational Geometry, pages 163-172, 1993.0 f[' €€€€‚ÿFW77›q+ @* $€â€Ȁ€€‚ÿS. Fiorini and R. Wilson. Edge-colourings of graphs. Research Notes in Mathematics 16, Pitman, London, 1977.[ @…ƒ0 [<@' €€€€‚ÿFW93·Œ @ó@+ $€€Ȁ€€‚ÿM. Fredman and D. Willard. Surpassing the information theoretic bound with fusion trees. J. Computer and System Sci., 47:424-436, 1993.1 <@$A' €€€€‚ÿGab76€zó@ÈA* $€ô€Ȁ€€‚ÿH. Gabow. An efficient implementation of Edmond's algorithm for maximum matching on graphs. J. ACM, 23:221-234, 1976.1 $AùA' €€€€‚ÿGab77œrÈA•B* $€ä€Ȁ€€‚ÿH. Gabow. Two algorithms for generating weighted spanning trees in order. SIAM J. Computing, 6:139-150, 1977.1 ùAÆB' €€€€‚ÿGal86s•BcC* $€æ€Ȁ€€‚ÿZ. Galil. Efficient algorithms for finding maximum matchings in graphs. ACM Computing Surveys, 18:23-38, 1986.2 ÆB•C' €€€€‚ÿGBDS80°…cCED+ $€ €Ȁ€€‚ÿB. Golden, L. Bodin, T. Doyle, and W. Stewart. Approximate traveling salesman algorithms. Operations Research, 28:694-711, 1980.1 •CvD' €€€€‚ÿGBY91³ˆED)E+ $€€Ȁ€€‚ÿG. Gonnet and R. Baeza-Yates. Handbook of Algorithms and Data Structures. Addison-Wesley, Wokingham, England, second edition, 1991.1 vDZE' €€€€‚ÿGGJ77®ƒ)EF+ $€€Ȁ€€‚ÿM. Garey, R. Graham, and D. Johnson. The complexity of computing Steiner minimal trees. SIAM J. Appl. Math., 32:835-859, 1977.2 ZE:F' €€€€‚ÿGGJK78Ž‰FîF+ $€€Ȁ€€‚ÿM. Garey, R. Graham, D. Johnson, and D. Knuth. Complexity results for bandwidth minimization. SIAM J. Appl. Math., 34:477-495, 1978.0 :FG' €€€€‚ÿGH85®ƒîFÌG+ $€€Ȁ€€‚ÿR. Graham and P. Hell. On the history of the minimum spanning tree problem. Annals of the History of Computing, 7:43-57, 1985.2 GþG' €€€€‚ÿGHMS93÷ÌÌGõH+ $€™€Ȁ€€‚ÿL. J. Guibas, J. E. Hershberger, J. S. B. Mitchell, and J. S. Snoeyink. Approximating polygons and subdivisions with minimum link paths. Internat. J. Comput. Geom. Appl., 3(4):383-415, December 1993.1 þG&I' €€€€‚ÿGHR95µŠõHÛI+ $€€Ȁ€€‚ÿR. Greenlaw, J. Hoover, and W. Ruzzo. Limits to Parallel Computation: P-completeness theory. Oxford University Press, New York, 1995.0 &I J' €€€€‚ÿGI89ŸuÛIªJ* $€ê€Ȁ€€‚ÿD. Gusfield and R. Irving. The Stable Marriage Problem: structure and algorithms. MIT Press, Cambridge MA, 1989.0 JÚJ' €€€€‚ÿGI91²‡ªJŒK+ $€€Ȁ€€‚ÿZ. Galil and G. Italiano. Data structures and algorithms for disjoint set union problems. ACM Computing Surveys, 23:319-344, 1991.1 ÚJœK' €€€€‚ÿGib76—mŒKTL* $€Ú€Ȁ€€‚ÿN. E. Gibbs. A hybrid profile reduction algorithm. ACM Trans. Math. Softw., 2(4):378-387, December 1976.0 œK„L' €€€€‚ÿGJ77£yTL'M* $€ò€Ȁ€€‚ÿM. Garey and D. Johnson. The rectilinear Steiner tree problem is NP-complete. SIAM J. Appl. Math., 32:826-834, 1977.0 „LWM' €€€€‚ÿGJ79º'MN+ $€€Ȁ€€‚ÿM. R. Garey and D. S. Johnson. Computers and Intractability: A Guide to the theory of NP-completeness. W. H. Freeman, San Francisco, 1979.2 WMCN' €€€€‚ÿGJPT78§}NêN* $€ú€Ȁ€€‚ÿM. Garey, D. Johnson, F. Preparata, and R. Tarjan. Triangulating a simple polygon. Info. Proc. Letters, 7:175-180, 1978.0 CNO' €€€€‚ÿGK79ùÎêN€+ $€€Ȁ€€‚ÿA. Goralcikiova and V. Konbek. A reduct and closure algorithm for graphs. In Mathematical Foundations of Computer Science, pages 301-307. Springer Verlag, Lecture Notes in Computer SciO€…ƒence V. 74, 1979.0 OO€' €€€€‚ÿGK93س€'% €g€Ȁ‚ÿA. Goldberg and R. Kennedy. An efficient cost scaling algorithm for the assignment problem. Technical Report 93-1481, Department of Computer Science, Stanford University, 1993.1 O€X' €€€€‚ÿGKK74ä¹'<‚+ $€s€Ȁ€€‚ÿF. Glover, D. Karney, and D. Klingman. Implementation and computational comparisons of primal-dual computer codes for minimum-cost network flow problems. Networks, 4:191-212, 1974.1 Xm‚' €€€€‚ÿGKP89c<‚ú‚* $€Æ€Ȁ€€‚ÿR. Graham, D. Knuth, and O. Patashnik. Concrete Mathematics. Addison-Wesley, Reading MA, 1989.2 m‚,ƒ' €€€€‚ÿGlo89a~Tú‚ªƒ* $€š€Ȁ€€‚ÿF. Glover. Tabu search - Part 1. ORSA Journal on Computing, 1(3):190-206, 1989.2 ,ƒ܃' €€€€‚ÿGlo89b{QªƒW„* $€¢€Ȁ€€‚ÿF. Glover. Tabu search - Part 2. ORSA Journal on Computing, 2(1):4-32, 1989.1 ܃ˆ„' €€€€‚ÿGlo90xNW„…* $€œ€Ȁ€€‚ÿF. Glover. Tabu search: A tutorial. Interfaces, Vol. 20, :4, 74-94, 1990.0 ˆ„0…' €€€€‚ÿGM86}S…­…* $€Š€Ȁ€€‚ÿG. Gonnet and J.I. Munro. Heaps on heaps. SIAM J. Computing, 15:964-971, 1986.1 0…ޅ' €€€€‚ÿGol89–l­…t†* $€Ø€Ȁ€€‚ÿD. E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, 1989.1 ޅ¥†' €€€€‚ÿGol92Ϫt†t‡% €U€Ȁ‚ÿA. Goldberg. An efficient implementation of a scaling minimum-cost flow algorithm. Technical Report 92-1439, Department of Computer Science, Stanford University, 1992.1 ¥†¥‡' €€€€‚ÿGol93˜nt‡=ˆ* $€Ü€Ȁ€€‚ÿL. Goldberg. Efficient Algorithms for Listing Combinatorial Structures. Cambridge University Press, 1993.0 ¥‡mˆ' €€€€‚ÿGP68…[=ˆòˆ* $€¶€Ȁ€€‚ÿE. Gilbert and H. Pollak. Steiner minimal trees. SIAM J. Applied Math., 16:1-29, 1968.0 mˆ"‰' €€€€‚ÿGP79›qòˆœ‰* $€â€Ȁ€€‚ÿB. Gates and C. Papadimitriou. Bounds for sorting by prefix reversals. Discrete Mathematics, 27:47-57, 1979.1 "‰î‰' €€€€‚ÿGPS76ƛœ‰ŽŠ+ $€7€Ȁ€€‚ÿN. Gibbs, W. Poole, and P. Stockmeyer. A comparison of several bandwidth and profile reduction algorithms. ACM Trans. Math. Software, 2:322-330, 1976.1 î‰åŠ' €€€€‚ÿGra53lHŽŠQ‹$ €€Ȁ‚ÿF. Gray. Pulse code communication. US Patent 2632058, March 17, 1953.1 劂‹' €€€€‚ÿGra72µŠQ‹7Œ+ $€€Ȁ€€‚ÿR. Graham. An efficient algorithm for determining the convex hull of a finite planar point set. Info. Proc. Letters, 1:132-133, 1972.0 ‚‹gŒ' €€€€‚ÿGS62 v7Œ* $€ì€Ȁ€€‚ÿD. Gale and L. Shapely. College admissions and the stability of marriages. American Math. Monthly, 69:9-14, 1962.0 gŒ7' €€€€‚ÿGS78؍ú+ $€1€Ȁ€€‚ÿL. Guibas and R. Sedgewick. A dichromatic framework for balanced trees. In Proc. 19th IEEE Symp. Foundations of Computer Science, pages 8-21, 1978.0 7*Ž' €€€€‚ÿGT88’húŒŽ* $€Ð€Ȁ€€‚ÿA. Goldberg and R. Tarjan. A new approach to the maximum flow problem. J. ACM, pages 921-940, 1988.0 *ŽìŽ' €€€€‚ÿGT89sŒŽ‰* $€æ€Ȁ€€‚ÿH. Gabow and R. Tarjan. Faster scaling algorithms for network problems. SIAM J. Computing, 18:1013-1036, 1989.1 쎺' €€€€‚ÿGup66s‰cÀ* $€æ€Ȁ€€‚ÿR. P. Gupta. The chromatic ºcÀ…ƒindex and the degree of a graph. Notices of the Amer. Math. Soc., 13:66T-429, 1966.1 º”À' €€€€‚ÿGus97pFcÀÁ* $€Œ€Ȁ€€‚ÿD. Gusfield. String Algorithms. Cambridge University Press, 1997.0 ”À4Á' €€€€‚ÿGW95 vÁÔÁ* $€ì€Ȁ€€‚ÿM. Goemans and D. Williamson. .878-approximation algorithms for MAX CUT and MAX 2SAT. J. ACM, 42:1115-1145, 1995.0 4ÁÂ' €€€€‚ÿHD80ŒbÔÁÂ* $€Ä€Ȁ€€‚ÿP. Hall and G. Dowling. Approximate string matching. ACM Computing Surveys, 12:381-402, 1980.0 ÂÀÂ' €€€€‚ÿHG95ÝžÂÃ% €q€Ȁ‚ÿP. Heckbert and M. Garland. Fast polygonal approximation of terrains and height fields. Technical Report CMU-CS-95-181, School of Computer Science, Carnegie-Mellon University, 1995.1 ÀÂÎÃ' €€€€‚ÿHim94ǜÃ•Ä+ $€9€Ȁ€€‚ÿM. Himsolt. GraphEd: A graphical platform for the implementation of graph algorithms. In Proc. Graph Drawing (GD '94), volume 894, pages 182-193, 1994.1 ÎÃÆÄ' €€€€‚ÿHir75¯„•ÄuÅ+ $€ €Ȁ€€‚ÿD. Hirschberg. A linear-space algorithm for computing maximum common subsequences. Communications of the ACM, 18:341-343, 1975.1 ÆÄŠÅ' €€€€‚ÿHJS84ŚuÅkÆ+ $€5€Ȁ€€‚ÿR. E. Haymond, J. P. Jarvis, and D. R. Shier. Minimum spanning tree for moderate integer weights. ACM Trans. Math. Softw., 10(1):108-111, March 1984.0 ŠÅ›Æ' €€€€‚ÿHK73®{kÆIÇ3 6€ø€Ȁ†"€€€‚ÿJ. Hopcroft and R. Karp. An algorithm for maximum matchings in bipartite graphs. SIAM J. Computing, 2:225-231, 1973.0 ›ÆyÇ' €€€€‚ÿHK90Þ³IÇWÈ+ $€g€Ȁ€€‚ÿD. P. Huttenlocher and K. Kedem. Computing the minimum Hausdorff distance for point sets under translation. In Proc. 6th Annu. ACM Sympos. Comput. Geom., pages 340-349, 1990.0 yLJÈ' €€€€‚ÿHM83çŒWÈnÉ+ $€y€Ȁ€€‚ÿS. Hertel and K. Mehlhorn. Fast triangulation of simple polygons. In Proc. 4th Internat. Conf. Found. Comput. Theory, pages 207-218. Lecture Notes in Computer Science, Vol. 158, 1983.2 ‡È É' €€€€‚ÿHMMN84Í¢nÉmÊ+ $€E€Ȁ€€‚ÿS. Hertel, K. Mehlhorn, M. Mäntylä, and J. Nievergelt. Space sweep solves intersection of two convex polyhedra elegantly. Acta Informatica, 21:501-519, 1984.1  ÉžÊ' €€€€‚ÿHoa61œrmÊ:Ë* $€ä€Ȁ€€‚ÿC. A. R. Hoare. Algorithm 63 (partition) and algorithm 65 (find). Communications of the ACM, 4:321-322, 1961.1 žÊkË' €€€€‚ÿHoa62j@:ËÕË* $€€€Ȁ€€‚ÿC. A. R. Hoare. Quicksort. Computer Journal, 5:10-15, 1962.1 kËÌ' €€€€‚ÿHoc96fÕË–Ì* $€Ì€Ȁ€€‚ÿD. Hochbaum, editor. Approximation Algorithms for NP-hard Problems. PWS Publishing, Boston, 1996.1 ÌÇÌ' €€€€‚ÿHof82П–Ì—Í1 0€?€Ȁ€€€€‚ÿC. M. Hoffmann. Group-theoretic algorithms and graph isomorphism, volume 136 of Lecture Notes in Computer Science. Springer-Verlag Inc., New York, 1982.1 ÇÌÈÍ' €€€€‚ÿHof89•k—Í]Î* $€Ö€Ȁ€€‚ÿC. Hoffman. The problem of accuracy and robustness in geometric computation. Computer, 22:31-42, 1989.1 ÈÍŽÎ' €€€€‚ÿHol75šp]Î(Ï* $€à€Ȁ€€‚ÿJ. H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, 1975.1 ŽÎYÏ' €€€€‚ÿHol81…[(ÏÞÏ* $€¶€Ȁ€€‚ÿI. Holyer. The NP-completeness of edge colorings. SIAM J. Computing, 10:718-720, 1981.1 YÏ ' €€€ÞÏ …ƒ€‚ÿHol92UÞÏš * $€ª€Ȁ€€‚ÿJ. H. Holland. Genetic algorithms. Scientific American, 267(1):66-72, July 1992.1  Ë ' €€€€‚ÿHop71ó¿š Ÿ 4 6€€Ȁ†"€€€‚ÿJ. Hopcroft. An algorithm for minimizing the states in a finite automaton. In Z. Kohavi, editor, The theory of machines and computations, pages 189-196. Academic Press, New York, 1971.1 Ë ï ' €€€€‚ÿHor80™oŸ ˆ * $€Þ€Ȁ€€‚ÿR. N. Horspool. Practical fast searching in strings. Software - Practice and Experience, 10:501-506, 1980.0 ï ž ' €€€€‚ÿHP73~Tˆ 6 * $€š€Ȁ€€‚ÿF. Harary and E. Palmer. Graphical enumeration. Academic Press, New York, 1973.1 ž g ' €€€€‚ÿHRW92ē6 + 1 0€'€Ȁ€€€€‚ÿR. Hwang, D. Richards, and P. Winter. The Steiner Tree Problem, volume 53 of Annals of Discrete Mathematics. North Holland, Amsterdam, 1992.0 g [ ' €€€€‚ÿHS77²‡+  + $€€Ȁ€€‚ÿJ. Hunt and T. Szymanski. A fast algorithm for computing longest common subsequences. Communications of the ACM, 20:350-353, 1977.0 [ = ' €€€€‚ÿHS94ê¶  ' 4 6€o€Ȁ†"€€€‚ÿJ. Hershberger and J. Snoeyink. An implementation of the Douglas-Peucker algorithm for line simplification. In Proc. 10th Annu. ACM Sympos. Comput. Geom., pages 383-384, 1994.1 = X ' €€€€‚ÿHSS87°…'  + $€ €Ȁ€€‚ÿJ. Hopcroft, J. Schwartz, and M. Sharir. Planning, geometry, and complexity of robot motion. Ablex Publishing, Norwood NJ, 1987.1 X 9 ' €€€€‚ÿHT73aœr Õ * $€ä€Ȁ€€‚ÿJ. Hopcroft and R. Tarjan. Dividing a graph into triconnected components. SIAM J. Computing, 2:135-158, 1973.1 9  ' €€€€‚ÿHT73b£yÕ © * $€ò€Ȁ€€‚ÿJ. Hopcroft and R. Tarjan. Efficient algorithms for graph manipulation. Communications of the ACM, 16:372-378, 1973.0  Ù ' €€€€‚ÿHT74€V© Y * $€¬€Ȁ€€‚ÿJ. Hopcroft and R. Tarjan. Efficient planarity testing. J. ACM, 21:549-568, 1974.0 Ù ‰ ' €€€€‚ÿHT84¢xY + * $€ð€Ȁ€€‚ÿD. Harel and R. E. Tarjan. Fast algorithms for finding nearest common ancestors. SIAM J. Comput., 13:338-355, 1984.1 ‰ \ ' €€€€‚ÿHua92•k+ ñ * $€Ö€Ȁ€€‚ÿX. Huang. A contig assembly program based on sensitive detection of fragment overlaps. Genomics, 1992.1 \ " ' €€€€‚ÿHua94pFñ ’ * $€Œ€Ȁ€€‚ÿX. Huang. On global sequence alignment. CABIOS, 10:227-235, 1994.1 " à ' €€€€‚ÿHuf52›q’ ^ * $€â€Ȁ€€‚ÿD. Huffman. A method for the construction of minimum-redundancy codes. Proc. of the IRE, 40:1098-1101, 1952.0 à Ž ' €€€€‚ÿHW74׬^ e + $€Y€Ȁ€€‚ÿJ. E. Hopcroft and J. K. Wong. Linear time algorithm for isomorphism of planar graphs. In Proc. Sixth Annual ACM Symposium on Theory of Computing, pages 172-184, 1974.1 Ž – ' €€€€‚ÿHWK94¡we 7 * $€î€Ȁ€€‚ÿT. He, S. Wang, and A. Kaufman. Wavelet-based volume morphing. In Proc. IEEE Visualization '94, pages 85-92, 1994.1 – h ' €€€€‚ÿHWL87”j7 ü * $€Ô€Ȁ€€‚ÿJr. H. W. Lenstra. Factoring integers with elliptic curves. Annals of Mathematics, 126:649-673, 1987.0 h , ' €€€€‚ÿIK75¥{ü Ñ * $€ö€Ȁ€€‚ÿO. Ibarra and C. Kim. Fast approximation algorithms for knapsack and sum of subset problems. J. ACM, 22:463-468, 1975.1 , @ ' €€€€‚ÿIta78Ñ @ …ƒe;Ñ s@ * $€v€Ȁ€€‚ÿA. Itai. Two commodity flow. J. ACM, 25:596-611, 1978.2 @ ¥@ ' €€€€‚ÿJAMS91 às@ °A + $€Á€Ȁ€€‚ÿD. Johnson, C. Aragon, C. McGeoch, and D. Schevon. Optimization by simulated annealing: an experimental evaluation; partII, graph coloring and number partitioning. In Operations Research, volume 39, pages 378-406, 1991.1 ¥@ áA ' €€€€‚ÿJan76«€°A ŒB + $€€Ȁ€€‚ÿW. Janko. A list insertion sort for keys with arbitrary key distribution. ACM Trans. Math. Softw., 2(2):204-206, June 1976.1 áA œB ' €€€€‚ÿJar73²‡ŒB oC + $€€Ȁ€€‚ÿR. A. Jarvis. On the identification of the convex hull of a finite set of points in the plane. Info. Proc. Letters, 2:18-21, 1973.0 œB ŸC ' €€€€‚ÿJD88ŽdoC -D * $€È€Ȁ€€‚ÿA. Jain and R. Dubes. Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs NJ, 1988.0 ŸC ]D ' €€€€‚ÿJM93Ô©-D 1E + $€S€Ȁ€€‚ÿD. Johnson and C. McGeoch, editors. Network Flows and Matching: First DIMACS Implementation Challenge, volume 12. American Mathematics Society, Providence RI, 1993.1 ]D bE ' €€€€‚ÿJoh63™o1E ûE * $€Þ€Ȁ€€‚ÿS. M. Johnson. Generation of permutations by adjacent transpositions. Math. Computation, 17:282-285, 1963.1 bE ,F ' €€€€‚ÿJoh74¡wûE ÍF * $€î€Ȁ€€‚ÿD. Johnson. Approximation algorithms for combinatorial problems. J. Computer and System Sciences, 9:256-278, 1974.1 ,F þF ' €€€€‚ÿJoh90èœÍF æG + $€{€Ȁ€€‚ÿD. S. Johnson. A catalog of complexity classes. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science: Algorithms and Complexity, volume A, pages 67-162. MIT Press, 1990.1 þF H ' €€€€‚ÿJon86²‡æG ÉH + $€€Ȁ€€‚ÿD. W. Jones. An empirical comparison of priority-queue and event-set implementations. Communications of the ACM, 29:300-311, 1986.1 H úH ' €€€€‚ÿKah67‚XÉH |I * $€°€Ȁ€€‚ÿD. Kahn. The Code breakers: the story of secret writing. Macmillan, New York, 1967.1 úH ­I ' €€€€‚ÿKah80Ÿ“|I kJ + $€'€Ȁ€€‚ÿD. A. Kahamer. Fortran implementation of heap programs for efficient table maintenance. ACM Trans. Math. Softw., 6(3):444-449, September 1980.1 ­I œJ ' €€€€‚ÿKar72Ô©kJ pK + $€S€Ȁ€€‚ÿR. M. Karp. Reducibility among combinatorial problems. In R. Miller and J. Thatcher, editors, Complexity of Computer Computations, pages 85-103. Plenum Press, 1972.1 œJ ¡K ' €€€€‚ÿKar84”jpK 5L * $€Ô€Ȁ€€‚ÿN. Karmarkar. A new polynomial-time algorithm for linear programming. Combinatorica, 4:373-395, 1984.2 ¡K gL ' €€€€‚ÿKar96a›q5L M * $€â€Ȁ€€‚ÿD. Karger. Minimum cuts in near-linear time. In Proc. 28th ACM Symp. Theory of Computing, pages 56-63, 1996.2 gL 4M ' €€€€‚ÿKar96bʟM þM + $€?€Ȁ€€‚ÿH. Karloff. How good is the Goemans-Williamson MAX CUT algorithm? In Proc. Twenty-Eighth Annual ACM Symposium on Theory of Computing, pages 427-434, 1996.1 4M /N ' €€€€‚ÿKei85‹aþM ºN * $€Â€Ȁ€€‚ÿM. Keil. Decomposing a polygon into simpler components. SIAM J. Computing, 14:799-817, 1985.1 /N ëN ' €€€€‚ÿKGV83€zºN O * $€ô€Ȁ€€‚ÿS. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi. Optimization by simulated annealing. Science, 220:671-680, 1983.0 ëN ¿O ' €€€€‚ÿKH80•kO `€ * $€Ö€Ȁ€€‚ÿJ. Kennington and R. H¿O `€ …ƒelgason. Algorithms for Network Programming. Wiley-Interscience, New York, 1980.1 ¿O ‘€ ' €€€€‚ÿKha79e`€  * $€Ê€Ȁ€€‚ÿL. Khachian. A polynomial algorithm in linear programming. Soviet Math. Dokl., 20:191-194, 1979.1 ‘€ Q ' €€€€‚ÿKir79Ž‰  ‚ + $€€Ȁ€€‚ÿD. Kirkpatrick. Efficient computation of continuous skeletons. In Proc. 20th IEEE Symp. Foundations of Computing, pages 28-35, 1979.1 Q 6‚ ' €€€€‚ÿKir83ˆ^‚ Ÿ‚ * $€Œ€Ȁ€€‚ÿD. Kirkpatrick. Optimal search in planar subdivisions. SIAM J. Computing, 12:28-35, 1983.1 6‚ ï‚ ' €€€€‚ÿKKT95°…Ÿ‚ Ÿƒ + $€ €Ȁ€€‚ÿD. Karger, P. Klein, and R. Tarjan. A randomized linear-time algorithm to find minimum spanning trees. J. ACM, 42:321-328, 1995.0 ï‚ Ïƒ ' €€€€‚ÿKL70œ’Ÿƒ Œ„ + $€%€Ȁ€€‚ÿB. W. Kernighan and S. Lin. An efficient heuristic procedure for partitioning graphs. The Bell System Technical Journal, pages 291-307, 1970.0 σ Œ„ ' €€€€‚ÿKM72¬Œ„ h… + $€€Ȁ€€‚ÿV. Klee and G. Minty. How good is the simplex algorithm. In Inequalities III, pages 159-172, New York, 1972. Academic Press.0 Œ„ ˜… ' €€€€‚ÿKM95¯„h… G† + $€ €Ȁ€€‚ÿJ. D. Kececioglu and E. W. Myers. Combinatorial algorithms for DNA sequence assembly. Algorithmica, 13(1/2):7-51, January 1995.1 ˜… x† ' €€€€‚ÿKMP77—mG† ‡ * $€Ú€Ȁ€€‚ÿD. Knuth, J. Morris, and V. Pratt. Fast pattern matching in strings. SIAM J. Computing, 6:323-350, 1977.1 x† @‡ ' €€€€‚ÿKno79›q‡ ۇ * $€â€Ȁ€€‚ÿH. D. Knoble. An efficient one-way enciphering algorithm. ACM Trans. Math. Softw., 5(1):108-111, March 1979.2 @‡ ˆ ' €€€€‚ÿKnu73a±†Û‡ Ÿˆ + $€ €Ȁ€€‚ÿD. E. Knuth. The Art of Computer Programming, Volume 1: Fundamental Algorithms. Addison-Wesley, Reading MA, second edition, 1973.2 ˆ ðˆ ' €€€€‚ÿKnu73bŸuŸˆ ‰ * $€ê€Ȁ€€‚ÿD. E. Knuth. The Art of Computer Programming, Volume 3: Sorting and Searching. Addison-Wesley, Reading MA, 1973.1 ðˆ À‰ ' €€€€‚ÿKnu81³ˆ‰ sŠ + $€€Ȁ€€‚ÿD. E. Knuth. The Art of Computer Programming, Volume 2: Seminumerical Algorithms. Addison-Wesley, Reading MA, second edition, 1981.1 À‰ €Š ' €€€€‚ÿKnu94“isŠ 7‹ * $€Ò€Ȁ€€‚ÿD. Knuth. The Stanford GraphBase: a platform for combinatorial computing. ACM Press, New York, 1994.0 €Š g‹ ' €€€€‚ÿKO63 v7‹ Œ * $€ì€Ȁ€€‚ÿA. Karatsuba and Yu. Ofman. Multiplication of multi-digit numbers on automata. Sov. Phys. Dokl., 7:595-596, 1963.1 g‹ 8Œ ' €€€€‚ÿKOS91ŚŒ ýŒ + $€5€Ȁ€€‚ÿA. Kaul, M. A. O'Connor, and V. Srinivasan. Computing Minkowski sums of regular polygons. In Proc. 3rd Canad. Conf. Comput. Geom., pages 74-77, 1991.1 8Œ . ' €€€€‚ÿKoz92oEýŒ  * $€Š€Ȁ€€‚ÿJ. R. Koza. Genetic Programming. MIT Press, Cambridge, MA, 1992.1 . ΍ ' €€€€‚ÿKPS89üѝ ʎ + $€£€Ȁ€€‚ÿB. Korte, H. Prömel, and A. Steger. Steiner trees in VLSI-layout. In B. Korte, L. Lovasz, H. Prömel, and A. Schrijver, editors, Paths, Flows, and VLSI-layout, pages 185-214. Springer-Verlag, Berlin, 1989.0 ΍ úŽ ' €€€€‚ÿKR87©ʎ £ * $€þ€Ȁ€€‚ÿR. Karp and M. Rabin. Efficient randomized pattern-matching algorithms. IBM J. Research and Development, 31:249-260, 1987.0 úŽ ӏ ' €€€€‚ÿKR91Š|£ …À * $€ø€Ȁ€€‚ÿA.ӏ …À …ƒ Kanevsky and V. Ramachandran. Improved algorithms for graph four-connectivity. J. Comp. Sys. Sci., 42:288-306, 1991.1 ӏ ¶À ' €€€€‚ÿKru56ɞ…À Á + $€=€Ȁ€€‚ÿJ. B. Kruskal. On the shortest spanning subtree of a graph and the traveling salesman problem. Proc. of the American Mathematical Society, 7:48-50, 1956.0 ¶À ¯Á ' €€€€‚ÿKS85·ŒÁ f + $€€Ȁ€€‚ÿM. Keil and J. R. Sack. Computational Geometry, chapter Minimum decomposition of geometric objects, pages 197-216. North-Holland, 1985.0 ¯Á – ' €€€€‚ÿKS86œrf 2à * $€ä€Ȁ€€‚ÿD. Kirkpatrick and R. Siedel. The ultimate planar convex hull algorithm? SIAM J. Computing, 15:287-299, 1986.0 – bà ' €€€€‚ÿKS90èœ2à JÄ + $€{€Ȁ€€‚ÿK. Kedem and M. Sharir. An efficient motion planning algorithm for a convex rigid polygonal object in 2-dimensional polygonal space. Discrete and Computational Geometry, 5:43-75, 1990.1 bà {Ä ' €€€€‚ÿKuh75ΣJÄ IÅ + $€G€Ȁ€€‚ÿH. W. Kuhn. Steiner's problem revisited. In G. Dantzig and B. Eaves, editors, Studies in Optimization, pages 53-70. Mathematical Association of America, 1975.1 {Ä zÅ ' €€€€‚ÿKur30ŽdIÅ Æ * $€È€Ȁ€€‚ÿK. Kuratowski. Sur le problème des courbes gauches en topologie. Fund. Math., 15:217-283, 1930.1 zÅ 9Æ ' €€€€‚ÿKwa62†\Æ ¿Æ * $€ž€Ȁ€€‚ÿM. Kwan. Graphic programming using odd and even points. Chinese Math., 1:273-277, 1962.1 9Æ ðÆ ' €€€€‚ÿLat91~T¿Æ nÇ * $€š€Ȁ€€‚ÿJ.-C. Latombe. Robot Motion Planning. Kluwer Academic Publishers, Boston, 1991.1 ðÆ ŸÇ ' €€€€‚ÿLaw76žtnÇ =È * $€è€Ȁ€€‚ÿE. Lawler. Combinatorial Optimization: Networks and Matroids. Holt, Rinehart, and Winston, Fort Worth TX, 1976.1 ŸÇ nÈ ' €€€€‚ÿLee82µŠ=È #É + $€€Ȁ€€‚ÿD. T. Lee. Medial axis transformation of a planar shape. IEEE Trans. Pattern Analysis and Machine Intelligence, PAMI-4:363-369, 1982.1 nÈ TÉ ' €€€€‚ÿLee94?#É “Ê + $€)€Ȁ€€‚ÿH. Leeb. pLab - a system for testing random numbers. In Proc. Int. Workshop Parallel Numerics, pages 89-99. Slovak Academy of Science, Institute for Informatics, Slovakia, 1994. Also available via ftp from http://random.mat.sbg.ac.at/ftp/pub/publications/leeb/smolenice/.1 TÉ ÄÊ ' €€€€‚ÿLen87°…“Ê tË + $€ €Ȁ€€‚ÿT. Lengauer. Efficient algorithms for finding minimum spanning forests of hierarchically defined graphs. J. Algorithms, 8, 1987.1 ÄÊ ¥Ë ' €€€€‚ÿLen89ˆ^tË -Ì * $€Œ€Ȁ€€‚ÿT. Lengauer. Hierarchical planarity testing algorithms. J. ACM, 36(3):474-509, July 1989.1 ¥Ë ^Ì ' €€€€‚ÿLen90•k-Ì óÌ * $€Ö€Ȁ€€‚ÿT. Lengauer. Combinatorial Algorithms for Integrated Circuit Layout. Wiley, Chichester, England, 1990.1 ^Ì $Í ' €€€€‚ÿLev92“ióÌ ·Í * $€Ò€Ȁ€€‚ÿJ. L. Leva. A normal random number generator. ACM Trans. Math. Softw., 18(4):454-455, December 1992.1 $Í èÍ ' €€€€‚ÿLew82Ø·Í «Î + $€1€Ȁ€€‚ÿJ. G. Lewis. The Gibbs-Poole-Stockmeyer and Gibbs-King algorithms for reordering sparse matrices. ACM Trans. Math. Softw., 8(2):190-194, June 1982.1 èÍ ÜÎ ' €€€€‚ÿLin65–l«Î rÏ * $€Ø€Ȁ€€‚ÿS. Lin. Computer solutions of the traveling salesman problem. Bell System Tech. J., 44:2245-2269, 1965.0 ÜÎ ¢Ï ' €€€€‚ÿLK73Ž‰rÏ b + $€€Ȁ€€‚ÿS. Lin and B. Kernighan. An effective heuristic al¢Ï b …ƒgorithm for the traveling salesman problem. Operations Research, 21:498-516, 1973.1 ¢Ï “ ' €€€€‚ÿLLK83›qb . * $€â€Ȁ€€‚ÿJ. K. Lenstra, E. L. Lawler, and A. Rinnooy Kan. Theory of Sequencing and Scheduling. Wiley, New York, 1983.2 “ ` ' €€€€‚ÿLLKS85–l. ö * $€Ø€Ȁ€€‚ÿE. Lawler, J. Lenstra, A. Rinnooy Kan, and D. Shmoys. The Traveling Salesman Problem. John Wiley, 1985.1 ` ' ' €€€€‚ÿLLS92ɞö ð + $€=€Ȁ€€‚ÿL. Lam, S.-W. Lee, and C. Suen. Thinning methodologies - a comprehensive survey. IEEE Trans. Pattern Analysis and Machine Intelligence, 14:869-885, 1992.0 '  ' €€€€‚ÿLP86yOð ™ * $€ž€Ȁ€€‚ÿL. Lovász and M. Plummer. Matching Theory. North-Holland, Amsterdam, 1986.1  Ê ' €€€€‚ÿLPW79µŠ™  + $€€Ȁ€€‚ÿT. Lozano-Perez and M. Wesley. An algorithm for planning collision-free paths among polygonal obstacles. Comm. ACM, 22:560-570, 1979.0 Ê ¯ ' €€€€‚ÿLR93Ö« … + $€W€Ȁ€€‚ÿK. Lang and S. Rao. Finding near-optimal cuts: An empirical evaluation. In Proc. 4th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA '93), pages 212-221, 1993.0 ¯ µ ' €€€€‚ÿLS87Õª… Š + $€U€Ȁ€€‚ÿV. Lumelski and A. Stepanov. Path planning strategies for a point mobile automaton moving amidst unknown obstacles of arbitrary shape. Algorithmica, 3:403-430, 1987.0 µ º ' €€€€‚ÿLS95œrŠ V * $€ä€Ȁ€€‚ÿY.-L. Lin and S. Skiena. Algorithms for square roots of graphs. SIAM J. Discrete Mathematics, 8:99-118, 1995.0 º † ' €€€€‚ÿLT79¥{V + * $€ö€Ȁ€€‚ÿR. Lipton and R. Tarjan. A separator theorem for planar graphs. SIAM Journal on Applied Mathematics, 36:346-358, 1979.0 † [ ' €€€€‚ÿLT80—m+ ò * $€Ú€Ȁ€€‚ÿR. Lipton and R. Tarjan. Applications of a planar separator theorem. SIAM J. Computing, 9:615-626, 1980.1 [ # ' €€€€‚ÿLuc91sIò – * $€’€Ȁ€€‚ÿE. Lucas. Récréations Mathématiques. Gauthier-Villares, Paris, 1891.1 # Ç ' €€€€‚ÿLuk80Ø­– Ÿ + $€[€Ȁ€€‚ÿE. M. Luks. Isomorphism of bounded valence can be tested in polynomial time. In Proc. of the 21st Annual Symposium on Foundations of Computing, pages 42-49. IEEE, 1980.0 Ç Ï ' €€€€‚ÿLV93£yŸ r * $€ò€Ȁ€€‚ÿM. Li and P. Vitáni. An introduction to Kolmogorov complexity and its applications. Springer-Verlag, New York, 1993.0 Ï ¢ ' €€€€‚ÿLW77æ»r ˆ + $€w€Ȁ€€‚ÿD. T. Lee and C. K. Wong. Worst-case analysis for region and partial region searches in multidimensional binary search trees and balanced quad trees. Acta Informatica, 9:23-29, 1977.0 ¢ ž ' €€€€‚ÿLW88Ÿ“ˆ v + $€'€Ȁ€€‚ÿT. Lengauer and E. Wanke. Efficient solution of connectivity problems on hierarchically defined graphs. SIAM J. Computing, 17:1063-1080, 1988.0 ž Š ' €€€€‚ÿLY93Ë v q + $€A€Ȁ€€‚ÿC. Lund and M. Yannakakis. On the hardness of approximating minimization problems. In Proc. 25th ACM Symp. Theory of Computing (STOC), pages 286-293, 1993.1 Š ¢ ' €€€€‚ÿMan89wMq  * $€š€Ȁ€€‚ÿU. Manber. Introduction to Algorithms. Addison-Wesley, Reading MA, 1989.1 ¢ J ' €€€€‚ÿMar83Ÿ“ @ + $€'€Ȁ€€‚ÿS. Martello. An enumerative algorithm for finding Hamiltonian circuits in a directed graph. ACM Trans. Math. Softw., 9(1):131-138, MarchJ @ …ƒ 1983.1 J E@ ' €€€€‚ÿMat87č@ A 7 <€€Ȁ€€€€€€‚ÿD. W. Matula. Determining edge connectivity in O(nm). In 28th Ann. Symp. Foundations of Computer Science, pages 249-251. IEEE, 1987.1 E@ :A ' €€€€‚ÿMcC76c A ÇA * $€Æ€Ȁ€€‚ÿE. McCreight. A space-economical suffix tree construction algorithm. J. ACM, 23:262-272, 1976.1 :A øA ' €€€€‚ÿMcH90}SÇA uB * $€Š€Ȁ€€‚ÿJ. McHugh. Algorithmic Graph Theory. Prentice-Hall, Englewood Cliffs NJ, 1990.1 øA ŠB ' €€€€‚ÿMcN83Š|uB LC * $€ø€Ȁ€€‚ÿJ. M. McNamee. A sparse matrix package - part II: Special cases. ACM Trans. Math. Softw., 9(3):344-345, September 1983.1 ŠB }C ' €€€€‚ÿMeg83°}LC -D 3 6€ü€Ȁ†"€€€‚ÿN. Megiddo. Linear time algorithm for linear programming in and related problems. SIAM J. Computing, 12:759-776, 1983.1 }C ^D ' €€€€‚ÿMeh84†\-D äD * $€ž€Ȁ€€‚ÿK. Mehlhorn. Data structures and algorithms, volume 1-3. Springer-Verlag, Berlin, 1984.1 ^D E ' €€€€‚ÿMen27vLäD ‹E * $€˜€Ȁ€€‚ÿK. Menger. Zur allgemeinen kurventheorie. Fund. Math., 10:96-115, 1927.1 E ŒE ' €€€€‚ÿMGH81Ö«‹E ’F + $€W€Ȁ€€‚ÿJ. J. Moré, B. S. Garbow, and K. E. Hillstrom. Fortran subroutines for testing unconstrained optimization software. ACM Trans. Math. Softw., 7(1):136-140, March 1981.0 ŒE ÂF ' €€€€‚ÿMH78Š|’F hG * $€ø€Ȁ€€‚ÿR. Merkle and M Hellman. Hiding and signatures in trapdoor knapsacks. IEEE Trans. Information Theory, 24:525-530, 1978.1 ÂF ™G ' €€€€‚ÿMic92‘ghG *H * $€Î€Ȁ€€‚ÿZ. Michalewicz. Genetic Algorithms + Data Structures = Evolution Programs. Springer, Berlin, 1992.1 ™G [H ' €€€€‚ÿMie58~T*H ÙH * $€š€Ȁ€€‚ÿW. Miehle. Link-minimization in networks. Operations Research, 6:232-243, 1958.1 [H I ' €€€€‚ÿMil76špÙH €I * $€à€Ȁ€€‚ÿG. Miller. Reimann's hypothesis and tests for primality. J. Computer and System Sciences, 13:300-317, 1976.1 I ÕI ' €€€€‚ÿMil89 á€I áJ + $€Ã€Ȁ€€‚ÿV. Milenkovic. Double precision geometry: a general technique for calculating line and segment intersections using rounded arithmetic. In Proc. 30th IEEE Symp. Foundations of Computer Science (FOCS), pages 500-505, 1989.1 ÕI K ' €€€€‚ÿMil97fáJ ¢K * $€Ì€Ȁ€€‚ÿV. Milenkovic. Multiple translational containment. part II: exact algorithms. Algorithmica, 1997.1 K ÓK ' €€€€‚ÿMin78«{¢K ~L 0 0€ö€Ȁ€€€€‚ÿH. Minc. Permanents, volume 6 of Encyclopedia of Mathematics and its Applications. Addison-Wesley, Reading MA, 1978.1 ÓK ¯L ' €€€€‚ÿMit961~L àM 1 0€€Ȁ€€€€‚ÿJ. Mitchell. Guillotine subdivisions approximate polygonal subdivisions: Part II - A simple polynomial-time approximation scheme for geometric k-MST, TSP, and related problems. University at Stony Brook, Part I appears in SODA'96, pp. 402-408, 1996.0 ¯L N ' €€€€‚ÿMM90Û°àM ëN + $€a€Ȁ€€‚ÿU. Manber and G. Myers. Suffix arrays: A new method for on-line string searches. In Proceedings First ACM-SIAM Symposium on Discrete Algorithms, pages 319-327. SIAM, 1990.1 N O ' €€€€‚ÿMMI72ɞëN € + $€=€Ȁ€€‚ÿD. Matula, G. Marble, and J. Isaacson. Graph coloring algorithms. In R. C. Read, editor, Graph Theory and Computing, pages 109-122. Academic Press, 1972.O € …ƒ0 O <€ ' €€€€‚ÿMN95±† € í€ + $€ €Ȁ€€‚ÿK. Mehlhorn and S. Näher. LEDA, a platform for combinatorial and geometric computing. Communications of the ACM, 38:96-102, 1995.0 <€  ' €€€€‚ÿMO63Ÿuí€ Œ * $€ê€Ȁ€€‚ÿL. E. Moses and R. V. Oakford. Tables of Random Permutations. Stanford University Press, Stanford, Calif., 1963.1  í ' €€€€‚ÿMoo59·ŒŒ €‚ + $€€Ȁ€€‚ÿE. F. Moore. The shortest path in a maze. In Proc. International Symp. Switching Theory, pages 285-292. Harvard University Press, 1959.0 í Ô‚ ' €€€€‚ÿMP78¬€‚ €ƒ + $€€Ȁ€€‚ÿD. Muller and F. Preparata. Finding the intersection of two convex polyhedra. Theoretical Computer Science, 7:217-236, 1978.0 Ԃ °ƒ ' €€€€‚ÿMP80²‡€ƒ b„ + $€€Ȁ€€‚ÿW. Masek and M. Paterson. A faster algorithm for computing string edit distances. J. Computer and System Sciences, 20:18-31, 1980.0 °ƒ ’„ ' €€€€‚ÿMR95cb„ … * $€Æ€Ȁ€€‚ÿR. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, New York, 1995.2 ’„ Q… ' €€€€‚ÿMRRT53蜅 9† + $€{€Ȁ€€‚ÿN. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, and A. H. Teller. Equation of state calculations by fast computing machines. Journal of Chemical Physics, 21(6):1087-1092, June 1953.0 Q… i† ' €€€€‚ÿMS91¡w9† ‡ * $€î€Ȁ€€‚ÿB. Moret and H. Shapiro. Algorithm from P to NP: Design and Efficiency. Benjamin/Cummings, Redwood City, CA, 1991.0 i† :‡ ' €€€€‚ÿMS93Ô© ‡ ˆ + $€S€Ȁ€€‚ÿM. Murphy and S. Skiena. Ranger: A tool for nearest neighbor search in high dimensions. In Proc. Ninth ACM Symposium on Computational Geometry, pages 403-404, 1993.1 :‡ ?ˆ ' €€€€‚ÿMS95aº•ˆ ùˆ % €+€Ȁ‚ÿD. Margaritis and S. Skiena. Reconstructing strings from substrings in rounds. Proc. 36th IEEE Symp. Foundations of Computer Science (FOCS), 1995.1 ?ˆ *‰ ' €€€€‚ÿMS95b®ƒùˆ ؉ + $€€Ȁ€€‚ÿJ. S. B. Mitchell and S. Suri. Separation and approximation of polyhedral objects. Comput. Geom. Theory Appl., 5:95-114, 1995.0 *‰ Š ' €€€€‚ÿMT85¬Ø‰ ŽŠ + $€€Ȁ€€‚ÿS. Martello and P. Toth. A program for the 0-1 multiple knapsack problem. ACM Trans. Math. Softw., 11(2):135-140, June 1985.0 Š äŠ ' €€€€‚ÿMT87ÑŽŠ æ‹ 1 0€£€Ȁ€€€€‚ÿS. Martello and P. Toth. Algorithms for knapsack problems. In S. Martello, editor, Surveys in Combinatorial Optimization, volume 31 of Annals of Discrete Mathematics, pages 213-258. North-Holland, 1987.1 äŠ Œ ' €€€€‚ÿMT90ašpæ‹ ±Œ * $€à€Ȁ€€‚ÿS. Martello and P. Toth. Knapsack problems: algorithms and computer implementations. Wiley, New York, 1990.1 Œ ⌠' €€€€‚ÿMT90b韱Œ ˍ + $€}€Ȁ€€‚ÿK. Mehlhorn and A. Tsakalidis. Data structures. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science: Algorithms and Complexity, volume A, pages 301-341. MIT Press, 1990.1 ⌠ü ' €€€€‚ÿMul94¡wˍ Ž * $€î€Ȁ€€‚ÿK. Mulmuley. Computational Geometry: an introduction through randomized algorithms. Prentice-Hall, New York, 1994.0 ü ͎ ' €€€€‚ÿMV80×£Ž € 4 6€I€Ȁ†"€€€‚ÿS. Micali and V. Vazirani. An O algorithm for finding maximum matchings in general graphs. In Proc. 21st. Symp. Foundations of Computing, pages 17-27, 1980.0 ͎ ԏ ' €€€€‚ÿMW93U€ _À * $€ª€Ȁ€€‚ÿJԏ _À …ƒ. More and S. Wright. Optimization Software Guide. SIAM, Philadelphia PA, 1993.1 ԏ À ' €€€€‚ÿMye86’b_À "Á 0 0€Ä€Ȁ€€€€‚ÿE. Myers. An O(n d) difference algorithm and its variations. Algorithmica, 1:514-534, 1986.0 À RÁ ' €€€€‚ÿNC88e"Á áÁ * $€Ê€Ȁ€€‚ÿT. Nishizeki and N. Chiba. Planar Graphs: Theory and Algorithms. North-Holland, Amsterdam, 1988.0 RÁ  ' €€€€‚ÿNH93ØáÁ Ô + $€1€Ȁ€€‚ÿJ. Nievergelt and K. Hinrichs. Algorithms and Data Structures with applications to graphics and geometry. Prentice Hall, Englewood Cliffs NJ, 1993.0  à ' €€€€‚ÿNU95¬‡Ô °Ã % €€Ȁ‚ÿStefan Näher and C. Uhrig. The LEDA user manual, version r3.2. Available by ftp from ftp.mpi-sb.mpg.de in directory /pub/LEDA, 1995.0 à àà ' €€€€‚ÿNW78³ˆ°Ã “Ä + $€€Ȁ€€‚ÿA. Nijenhuis and H. Wilf. Combinatorial Algorithms for Computers and Calculators. Academic Press, Orlando FL, second edition, 1978.0 àà ÃÄ ' €€€€‚ÿNW86Ø­“Ä ›Å + $€[€Ȁ€€‚ÿJ. C. Nash and R. L. C. Wang. Subroutines for testing programs that compute the generalized inverse of a matrix. ACM Trans. Math. Softw., 12(3):274-277, September 1986.0 ÃÄ ËÅ ' €€€€‚ÿNZ80šp›Å eÆ * $€à€Ȁ€€‚ÿI. Niven and H. Zuckerman. An Introduction to the Theory of Numbers. Wiley, New York, fourth edition, 1980.1 ËÅ –Æ ' €€€€‚ÿOgn93‡]eÆ Ç * $€º€Ȁ€€‚ÿR. Ogniewicz. Discrete Voronoi Skeletons. Hartung-Gorre Verlag, Konstanz, Germany, 1993.1 –Æ NÇ ' €€€€‚ÿO'R87‡]Ç ÕÇ * $€º€Ȁ€€‚ÿJ. O'Rourke. Art Gallery Theorems and Algorithms. Oxford University Press, Oxford, 1987.1 NÇ È ' €€€€‚ÿO'R94„ZÕÇ ŠÈ * $€Ž€Ȁ€€‚ÿJ. O'Rourke. Computational Geometry in C. Cambridge University Press, New York, 1994.1 È »È ' €€€€‚ÿOrt88‘gŠÈ LÉ * $€Î€Ȁ€€‚ÿJ. Ortega. Introduction to Parallel and Vector Solution of Linear Systems. Plenum, New York, 1988.1 »È }É ' €€€€‚ÿOvL81®ƒLÉ +Ê + $€€Ȁ€€‚ÿM. Overmars and J. van Leeuwen. Maintenance of configurations in the plane. J. Computer and System Sciences, 23:166-204, 1981.0 }É [Ê ' €€€€‚ÿOW85ߎ+Ê :Ë + $€i€Ȁ€€‚ÿJ. O'Rourke and R. Washington. Curve similarity via signatures. In G. T. Toussaint, editor, Computational Geometry, pages 295-317. North-Holland, Amsterdam, Netherlands, 1985.1 [Ê kË ' €€€€‚ÿPag74Œb:Ë ÷Ë * $€Ä€Ȁ€€‚ÿR. L. Page. A minimal spanning tree clustering method. Commun. ACM, 17(6):321-323, June 1974.1 kË (Ì ' €€€€‚ÿPal85¥{÷Ë ÍÌ * $€ö€Ȁ€€‚ÿE. M. Palmer. Graphical Evolution: An Introduction to the Theory of Random Graphs. Wiley-Interscience, New York, 1985.2 (Ì ÿÌ ' €€€€‚ÿPap76a}SÍÌ |Í * $€Š€Ȁ€€‚ÿC. Papadimitriou. The complexity of edge traversing. J. ACM, 23:544-554, 1976.2 ÿÌ ®Í ' €€€€‚ÿPap76b˜n|Í FÎ * $€Ü€Ȁ€€‚ÿC. Papadimitriou. The NP-completeness of the bandwidth minimization problem. Computing, 16:263-270, 1976.1 ®Í wÎ ' €€€€‚ÿPap80…[FÎ üÎ * $€¶€Ȁ€€‚ÿU. Pape. Shortest path lengths. ACM Trans. Math. Softw., 6(3):450-455, September 1980.1 wÎ -Ï ' €€€€‚ÿPar90sIüÎ  Ï * $€’€Ȁ€€‚ÿG. Parker. A better phonetic search. C Gazette, 5-4, June/July 1990.1 -Ï ÑÏ ' €€€€‚ÿPav82•k Ï r * $€Ö€Ȁ€€‚ÿT. PÑÏ r …ƒavlidis. Algorithms for Graphics and Image Processing. Computer Science Press, Rockville MD, 1982.2 ÑÏ € ' €€€€‚ÿPFTV86À•r d + $€+€Ȁ€€‚ÿW. Press, B. Flannery, S. Teukolsky, and W. T. Vetterling. Numerical Recipes: the art of scientific computing. Cambridge University Press, 1986.1 € • ' €€€€‚ÿPGD82špd / * $€à€Ȁ€€‚ÿD. Phillips and A. Garcia-Diaz. Fundamentals of Network Analysis. Prentice-Hall, Englewood Cliffs NJ, 1982.0 • _ ' €€€€‚ÿPH80·Œ/  + $€€Ȁ€€‚ÿM. Padberg and S. Hong. On the symmetric traveling salesman problem: a computational study. Math. Programming Studies, 12:78-107, 1980.0 _ F ' €€€€‚ÿPL94ôÉ : + $€“€Ȁ€€‚ÿP. A. Pevzner and R. J. Lipshutz. Towards DNA sequencing chips. In 19th Int. Conf. Mathematical Foundations of Computer Science, volume 841, pages 143-158, Lecture Notes in Computer Science, 1994.0 F j ' €€€€‚ÿPM88«€:  + $€€Ȁ€€‚ÿS. Park and K. Miller. Random number generators: Good ones are hard to find. Communications of the ACM, 31:1192-1201, 1988.1 j F ' €€€€‚ÿPol57‰_ Ï * $€Ÿ€Ȁ€€‚ÿG. Polya. How to Solve it. Princeton University Press, Princeton NJ, second edition, 1957.1 F  ' €€€€‚ÿPom84ÙÏ  + $€³€Ȁ€€‚ÿC. Pomerance. The quadratic sieve factoring algorithm. In T. Beth, N. Cot, and I. Ingemarrson, editors, Advances in Cryptology, volume 209, pages 169-182. Lecture Notes in Computer Science, Springer-Verlag, 1984.0  4 ' €€€€‚ÿPR79£y × * $€ò€Ȁ€€‚ÿS. Peleg and A. Rosenfield. Breaking substitution ciphers using a relaxation algorithm. Comm. ACM, 22:598-605, 1979.1 4  ' €€€€‚ÿPra75„Z× Œ * $€Ž€Ȁ€€‚ÿV. Pratt. Every prime has a succinct certificate. SIAM J. Computing, 4:214-220, 1975.1  œ ' €€€€‚ÿPri57€zŒ a * $€ô€Ȁ€€‚ÿR. C. Prim. Shortest connection networks and some generalizations. Bell System Technical Journal, 36:1389-1401, 1957.1 œ ’ ' €€€€‚ÿPrü18Œba  * $€Ä€Ȁ€€‚ÿH. Prüfer. Neuer beweis eines satzes über permutationen. Arch. Math. Phys., 27:742-744, 1918.0 ’ N ' €€€€‚ÿPS82Ž‰  + $€€Ȁ€€‚ÿC. Papadimitriou and K. Steiglitz. Combinatorial Optimization: Algorithms and Complexity. Prentice-Hall, Englewood Cliffs, NJ, 1982.0 N 2 ' €€€€‚ÿPS85ƒY µ * $€²€Ȁ€€‚ÿF. Preparata and M. Shamos. Computational Geometry. Springer-Verlag, New York, 1985.1 2 æ ' €€€€‚ÿPSW92§}µ  * $€ú€Ȁ€€‚ÿT. Pavlides, J. Swartz, and Y. Wang. Information encoding with two-dimensional bar-codes. IEEE Computer, 25:18-28, 1992.1 æ Ÿ ' €€€€‚ÿPug90Ÿu ] * $€ê€Ȁ€€‚ÿW. Pugh. Skip lists: A probabilistic alternative to balanced trees. Communications of the ACM, 33:668-676, 1990.0 Ÿ  ' €€€€‚ÿPW83•k] " * $€Ö€Ȁ€€‚ÿS. Pizer and V. Wallace. To compute numerically: concepts and strategies. Little, Brown, Boston, 1983.1  S ' €€€€‚ÿRab80‹a" Þ * $€Â€Ȁ€€‚ÿM. Rabin. Probabilistic algorithm for testing primality. J. Number Theory, 12:128-138, 1980.1 S  ' €€€€‚ÿRab95µŠÞ Ä + $€€Ȁ€€‚ÿF. M. Rabinowitz. A stochastic algorithm for global optimization with constraints. ACM Trans. Math. Softw., 21(2):194-213, June 1995.1  @ ' €€€€‚ÿRaw92Ä @ …ƒtJÄ €@ * $€”€Ȁ€€‚ÿG. Rawlins. Compared to What? Computer Science Press, New York, 1992.0 @ °@ ' €€€€‚ÿRC55™o€@ IA * $€Þ€Ȁ€€‚ÿRand-Corporation. A million random digits with 100,000 normal deviates. The Free Press, Glencoe, IL, 1955.1 °@ zA ' €€€€‚ÿRDC93ɞIA CB + $€=€Ȁ€€‚ÿE. Reingold, N. Dershowitz, and S. Clamen. Calendrical calculations II: Three historical calendars. Software - Practice and Experience, 22:383-404, 1993.1 zA tB ' €€€€‚ÿRei72UCB óB * $€ª€Ȁ€€‚ÿE. Reingold. On the optimality of some set algorithms. J. ACM, 19:649-659, 1972.1 tB $C ' €€€€‚ÿRei91cóB ±C * $€Æ€Ȁ€€‚ÿG. Reinelt. TSPLIB - a traveling salesman problem library. ORSA J. Computing, 3:376-384, 1991.1 $C âC ' €€€€‚ÿRei94ᶱC ÃD + $€m€Ȁ€€‚ÿG. Reinelt. The traveling salesman problem: Computational solutions for TSP applications. In Lecture Notes in Computer Science 840, pages 172-186. Springer-Verlag, Berlin, 1994.1 âC ôD ' €€€€‚ÿRen84ØÃD ·E + $€1€Ȁ€€‚ÿR. J. Renka. Triangulation and interpolation at arbitrarily distributed points in the plane. ACM Trans. Math. Softw., 10(4):440-442, December 1984.1 ôD èE ' €€€€‚ÿRHS89 v·E ˆF * $€ì€Ȁ€€‚ÿA. Robison, B. Hafner, and S. Skiena. Eight pieces cannot cover a chessboard. Computer Journal, 32:567-570, 1989.1 èE ¹F ' €€€€‚ÿRiv77˜nˆF QG * $€Ü€Ȁ€€‚ÿR. Rivest. On the worst-case behavior of string-searching algorithms. SIAM J. Computing, 6:669-674, 1977.1 ¹F ‚G ' €€€€‚ÿRiv92c?QG åG $ €~€Ȁ‚ÿR. Rivest. The MD5 message digest algorithm. RFC 1321, 1992.1 ‚G H ' €€€€‚ÿRND77°…åG ÆH + $€ €Ȁ€€‚ÿE. Reingold, J. Nievergelt, and N. Deo. Combinatorial algorithms: theory and practice. Prentice-Hall, Englewood Cliffs NJ, 1977.0 H öH ' €€€€‚ÿRS96¿”ÆH µI + $€)€Ȁ€€‚ÿH. Rau and S. Skiena. Dialing for documents: an experiment in information theory. Journal of Visual Languages and Computing, pages 79-95, 1996.1 öH æI ' €€€€‚ÿRSA78ʟµI °J + $€?€Ȁ€€‚ÿR. Rivest, A. Shamir, and L. Adleman. A method for obtaining digital signatures and public-key cryptosystems. Communications of the ACM, 21:120-126, 1978.1 æI áJ ' €€€€‚ÿRSL77ƛ°J §K + $€7€Ȁ€€‚ÿD. Rosenkrantz, R. Stearns, and P. M. Lewis. An analysis of several heuristics for the traveling salesman problem. SIAM J. Computing, 6:563-581, 1977.2 áJ ÙK ' €€€€‚ÿRSST96Ï€§K šL + $€I€Ȁ€€‚ÿN. Robertson, D. Sanders, P. Seymour, and R. Thomas. Efficiently four-coloring planar graphs. In Proc. 28th ACM Symp. Theory of Computing, pages 571-575, 1996.0 ÙK ØL ' €€€€‚ÿRT81—mšL oM * $€Ú€Ȁ€€‚ÿE. Reingold and J. Tilford. Tidier drawings of trees. IEEE Trans. Software Engineering, 7:223-228, 1981.1 ØL  M ' €€€€‚ÿRus97fBoM N $ €„€Ȁ‚ÿF. Ruskey. Combinatorial generation. book in preparation, 1997.1  M 7N ' €€€€‚ÿRyt85¥{N ÜN * $€ö€Ȁ€€‚ÿW. Rytter. Fast recognition of pushdown automaton and context-free languages. Information and Control, 67:12-22, 1985.0 7N O ' €€€€‚ÿSA95Ž‰ÜN ÀO + $€€Ȁ€€‚ÿM. Sharir and P. Agarwal. Davenport-Schinzel sequences and their geometric applications. Cambridge University Press, New York, 1995.2 O € ' €€€€‚ÿSam90aÀO € …ƒƒYÀO € * $€²€Ȁ€€‚ÿH. Samet. Applications of spatial data structures. Addison-Wesley, Reading MA, 1990.2 € Á€ ' €€€€‚ÿSam90bŽd€ O * $€È€Ȁ€€‚ÿH. Samet. The design and analysis of spatial data structures. Addison-Wesley, Reading MA, 1990.1 Á€ € ' €€€€‚ÿSax80ΣO N‚ + $€G€Ȁ€€‚ÿJ. B. Saxe. Dynamic programming algorithms for recognizing small-bandwidth graphs in polynomial time. SIAM J. Algebraic and Discrete Methods, 1:363-369, 1980.1 € ‚ ' €€€€‚ÿSch94h>N‚ ç‚ * $€|€Ȁ€€‚ÿB. Schneier. Applied Cryptography. Wiley, New York, 1994.0 ‚ ƒ ' €€€€‚ÿSD75š~ç‚ ¿ƒ * $€ü€Ȁ€€‚ÿM. Syslo and J. Dzikiewicz. Computational experiences with some transitive closure algorithms. Computing, 15:33-39, 1975.0 ƒ ïƒ ' €€€€‚ÿSD76ɞ¿ƒ ž„ + $€=€Ȁ€€‚ÿD. C. Schmidt and L. E. Druffel. A fast backtracking algorithm to test directed graphs for isomorphism using distance matrices. J. ACM, 23:433-445, 1976.1 ïƒ é„ ' €€€€‚ÿSDK83²‡ž„ ›… + $€€Ȁ€€‚ÿM. Syslo, N. Deo, and J. Kowalik. Discrete Optimization Algorithms with Pascal Programs. Prentice Hall, Englewood Cliffs NJ, 1983.1 é„ Ì… ' €€€€‚ÿSed77€V›… L† * $€¬€Ȁ€€‚ÿR. Sedgewick. Permutation generation methods. Computing Surveys, 9:137-164, 1977.1 ̅ }† ' €€€€‚ÿSed78Š`L† ‡ * $€À€Ȁ€€‚ÿR. Sedgewick. Implementing quicksort programs. Communications of the ACM, 21:847-857, 1978.1 }† 8‡ ' €€€€‚ÿSed92qG‡ ©‡ * $€Ž€Ȁ€€‚ÿR. Sedgewick. Algorithms in C++. Addison-Wesley, Reading MA, 1992.0 8‡ ه ' €€€€‚ÿSF92൩‡ ¹ˆ + $€k€Ȁ€€‚ÿT. Schlick and A. Fogelson. TNPACK - a truncated Newton minimization package for large-scale problems: I. algorithm and usage. ACM Trans. Math. Softw., 18(1):46-70, March 1992.1 ه êˆ ' €€€€‚ÿSFG82 v¹ˆ Š‰ * $€ì€Ȁ€€‚ÿM. Shore, L. Foulds, and P. Gibbons. An algorithms for the Steiner problem in graphs. Networks, 12:323-333, 1982.0 êˆ º‰ ' €€€€‚ÿSH75²‡Š‰ lŠ + $€€Ȁ€€‚ÿM. Shamos and D. Hoey. Closest point problems. In Proc. Sixteenth IEEE Symp. Foundations of Computer Science, pages 151-162, 1975.0 º‰ œŠ ' €€€€‚ÿSH76¶‹lŠ R‹ + $€€Ȁ€€‚ÿM. Shamos and D. Hoey. Geometric intersection problems. In Proc. 17th IEEE Symp. Foundations of Computer Science, pages 208-215, 1976.1 œŠ ƒ‹ ' €€€€‚ÿSha78‚XR‹ Œ * $€°€Ȁ€€‚ÿM. Shamos. Computational Geometry. PhD thesis, Yale University, UMI #7819047, 1978.1 ƒ‹ 6Œ ' €€€€‚ÿSha87蜌  + $€{€Ȁ€€‚ÿM. Sharir. Efficient algorithms for planning purely translational collision-free motion in two and three dimensions. In Proc. IEEE Internat. Conf. Robot. Autom., pages 1326-1331, 1987.1 6Œ O ' €€€€‚ÿSha93§} ö * $€ú€Ȁ€€‚ÿR. Sharda. Linear and Discrete Optimization and Modeling Software: A Resource Handbook. Lionheart Publishing Inc., 1993.1 O 'Ž ' €€€€‚ÿShe78ǜö îŽ + $€9€Ȁ€€‚ÿA. H. Sherman. NSPIV: a Fortran subroutine for sparse Gaussian elimination with partial pivoting. ACM Trans. Math. Softw., 4(4):391-398, December 1978.1 'Ž  ' €€€€‚ÿShe96³ˆîŽ ҏ + $€€Ȁ€€‚ÿJ. R. Shewchuk. Robust adaptive floating-point geometric predicates. In Proc. 12th ACM Computational Geometry, pages 141-150, 1996.0  À ' €€€€‚ÿSK83ҏ À …ƒʟҏ ØÀ + $€?€Ȁ€€‚ÿD. Sankoff and J. Kruskal. Time Warps, String Edits, and Macromolecules: the theory and practice of sequence comparison. Addison-Wesley, Reading MA, 1983.0 À Á ' €€€€‚ÿSK86uKØÀ }Á * $€–€Ȁ€€‚ÿT. Saaty and P. Kainen. The Four-Color Problem. Dover, New York, 1986.0 Á ­Á ' €€€€‚ÿSK93”j}Á A * $€Ô€Ȁ€€‚ÿR. Skeel and J. Keiper. Elementary Numerical computing with Mathematica. McGraw-Hill, New York, 1993.1 ­Á r ' €€€€‚ÿSki88WA ó * $€®€Ȁ€€‚ÿS. Skiena. Encroaching lists as a measure of presortedness. BIT, 28:775-784, 1988.1 r $à ' €€€€‚ÿSki90„Zó šÃ * $€Ž€Ȁ€€‚ÿS. Skiena. Implementing Discrete Mathematics. Addison-Wesley, Redwood City, CA, 1990.1 $à Ùà ' €€€€‚ÿSla61ŒbšÃ eÄ * $€Ä€Ȁ€€‚ÿP. Slater. Inconsistencies in a schedule of paired comparisons. Biometrika, 48:303-312, 1961.0 Ùà •Ä ' €€€€‚ÿSM73Ž‰eÄ IÅ + $€€Ȁ€€‚ÿL. Stockmeyer and A. Meyer. Word problems requiring exponential time. In Proc. Fifth ACM Symp. Theory of Computing, pages 1-9, 1973.1 •Ä zÅ ' €€€€‚ÿSmi91³ˆIÅ -Æ + $€€Ȁ€€‚ÿD. M. Smith. A Fortran package for floating-point multiple-precision arithmetic. ACM Trans. Math. Softw., 17(2):273-283, June 1991.1 zÅ ^Æ ' €€€€‚ÿSou96·Œ-Æ Ç + $€€Ȁ€€‚ÿE. Southern. DNA chips: analysing sequence by hybridization to oligonucleotides on a large scale. Trends in Genetics, 12:110-115, 1996.1 ^Æ FÇ ' €€€€‚ÿSPP76¢xÇ èÇ * $€ð€Ȁ€€‚ÿA. Schönhage, M. Paterson, and N. Pippenger. Finding the median. J. Computer and System Sciences, 13:184-199, 1976.0 FÇ È ' €€€€‚ÿSR83–lèÇ ®È * $€Ø€Ȁ€€‚ÿK. Supowit and E. Reingold. The complexity of drawing trees nicely. Acta Informatica, 18:377-392, 1983.0 È ÞÈ ' €€€€‚ÿSR95‹a®È iÉ * $€Â€Ȁ€€‚ÿR. Sharda and G. Rampal. Algebraic modeling languages on PCs. OR/MS Today, 22-3:58-63, 1995.0 ÞÈ ™É ' €€€€‚ÿSS71eiÉ (Ê * $€Ê€Ȁ€€‚ÿA. Schönhage and V. Strassen. Schnele multiplikation grosser zahlen. Computing, 7:281-292, 1971.0 ™É XÊ ' €€€€‚ÿSS90ôÉ(Ê LË + $€“€Ȁ€€‚ÿJ. T. Schwartz and M. Sharir. Algorithmic motion planning. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science: Algorithms and Complexity, volume A, pages 391-430. MIT Press, 1990.1 XÊ }Ë ' €€€€‚ÿSSS74¶‹LË 3Ì + $€€Ȁ€€‚ÿI. Sutherland, R. Sproull, and R. Shumacker. A characterization of ten hidden surface algorithms. ACM Computing Surveys, 6:1-55, 1974.0 }Ë cÌ ' €€€€‚ÿST85†\3Ì éÌ * $€ž€Ȁ€€‚ÿD. Sleator and R. Tarjan. Self-adjusting binary search trees. J. ACM, 32:652-686, 1985.1 cÌ Í ' €€€€‚ÿSta90Š`éÌ €Í * $€À€Ȁ€€‚ÿJ. Stasko. Tango: A framework and system for algorithm animation. Computer, 23:27-39, 1990.1 Í ÕÍ ' €€€€‚ÿSta95—m€Í lÎ * $€Ú€Ȁ€€‚ÿW. Stallings. Protect your Privacy: a guide for PGP Users. Prentice Hall PTR, Englewood Cliffs NJ, 1995.1 ÕÍ Î ' €€€€‚ÿSte94š~lÎ EÏ * $€ü€Ȁ€€‚ÿG. A. Stephen. String Searching Algorithms. Lecture-Notes-Series-on-Computing. World-Scientific-Publishing, October 1994.1 Î vÏ ' €€€€‚ÿSto88‹aEÏ * $€Â€Ȁ€€‚ÿJ. Storer. Data compression: methods and theory. Computer Science Press, Rockville MD, 1988.vÏ …ƒ1 vÏ > ' €€€€‚ÿStr69‰_ Ç * $€Ÿ€Ȁ€€‚ÿV. Strassen. Gaussian elimination is not optimal. Numerische Mathematik, 14:354-356, 1969.0 > ÷ ' €€€€‚ÿSV87 vÇ — * $€ì€Ȁ€€‚ÿJ. Stasko and J. Vitter. Pairing heaps: Experiments and analysis. Communications of the ACM, 30(3):234-249, 1987.0 ÷ Ç ' €€€€‚ÿSV88ę— ‹ + $€3€Ȁ€€‚ÿB. Schieber and U. Vishkin. On finding lowest common ancestors: simplification and parallelization. SIAM J. Comput., 17(6):1253-1262, December 1988.0 Ç » ' €€€€‚ÿSW86„Z‹ ? * $€Ž€Ȁ€€‚ÿD. Stanton and D. White. Constructive Combinatorics. Springer-Verlag, New York, 1986.0 » o ' €€€€‚ÿSW94Û°? J + $€a€Ȁ€€‚ÿM. Stoer and F. Wagner. A simple min cut algorithm. In European Symp. Algorithms (ESA), volume 855, pages 141-147. Springer Verlag Lecture Notes in Computer Science, 1994.0 o z ' €€€€‚ÿSW95³ˆJ - + $€€Ȁ€€‚ÿJ. Salowe and D. Warme. Thirty-five-point rectilinear Steiner minimal trees in a day. Networks: An International Journal, 25, 1995.1 z ^ ' €€€€‚ÿSWM95ž-  + $€€Ȁ€€‚ÿJ. Shallit, H. Williams, and F. Moraine. Discovery of a lost factoring machine. The Mathematical Intelligencer, 17-3:41-47, Summer 1995.1 ^ G ' €€€€‚ÿTar95{Q  * $€¢€Ȁ€€‚ÿG. Tarry. Le problème de labyrinthes. Nouvelles Ann. de Math., 14:187, 1895.1 G ó ' €€€€‚ÿTar72c € * $€Æ€Ȁ€€‚ÿR. Tarjan. Depth-first search and linear graph algorithms. SIAM J. Computing, 1:146-160, 1972.1 ó ± ' €€€€‚ÿTar75‹a€ < * $€Â€Ȁ€€‚ÿR. Tarjan. Efficiency of a good by not linear set union algorithm. J. ACM, 22:215-225, 1975.1 ± m ' €€€€‚ÿTar79Œ‘< ) + $€#€Ȁ€€‚ÿR. Tarjan. A class of algorithms which require non-linear time to maintain disjoint sets. J. Computer and System Sciences, 18:110-127, 1979.1 m Z ' €€€€‚ÿTar83¥{) ÿ * $€ö€Ȁ€€‚ÿR. Tarjan. Data Structures and Network Algorithms. Society for Industrial and Applied Mathematics, Philadelphia, 1983.1 Z 0 ' €€€€‚ÿTho68cÿ œ * $€Æ€Ȁ€€‚ÿK. Thompson. Regular expression search algorithm. Communications of the ACM, 11:419-422, 1968.1 0 î ' €€€€‚ÿTho96¬œ š + $€€Ȁ€€‚ÿMikkel Thorup. On RAM priority queues. In Proc. Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, pages 59-67, 1996.1 î Ë ' €€€€‚ÿTin90~Tš I * $€š€Ȁ€€‚ÿG. Tinhofer. Generating graphs uniformly at random. Computing, 7:235-255, 1990.1 Ë z ' €€€€‚ÿTro62oEI é * $€Š€Ȁ€€‚ÿH. F. Trotter. Perm (algorithm 115). Comm. ACM, 5:434-435, 1962.1 z  ' €€€€‚ÿTur88`é ª 0 0€À€Ȁ€€€€‚ÿJ. Turner. Almost all k-colorable graphs are easy to color. J. Algorithms, 9:63-82, 1988.0  Ú ' €€€€‚ÿTW88«xª … 3 6€ò€Ȁ†"€ €€‚ÿR. Tarjan and C. Van Wyk. An O algorithm for triangulating a simple polygon. SIAM J. Computing, 17:143-178, 1988.1 Ú ¶ ' €€€€‚ÿUkk92Ÿ“… t + $€'€Ȁ€€‚ÿE. Ukkonen. Constructing suffix trees on-line in linear time. In Intern. Federation of Information Processing (IFIP '92), pages 484-492, 1992.1 ¶ ¥ ' €€€€‚ÿVai88—mt H@ * $€Ú€Ȁ€€‚ÿP. Vaidya. Geometry helps in matching. In Proc.¥ H@ …ƒ 20th ACM Symp. Theory of Computing, pages 422-425, 1988.1 ¥ y@ ' €€€€‚ÿVal79”jH@ A * $€Ô€Ȁ€€‚ÿL. Valiant. The complexity of computing the permanent. Theoretical Computer Science, 8:189-201, 1979.1 y@ >A ' €€€€‚ÿVar91‰_ A ÇA * $€Ÿ€Ȁ€€‚ÿI. Vardi. Computational Recreations in Mathematica. Addison-Wesley, Redwood City CA, 1991.1 >A øA ' €€€€‚ÿVau80ˆ^ÇA €B * $€Œ€Ȁ€€‚ÿJ. Vaucher. Pretty printing of trees. Software Practice and Experience, 10:553-561, 1980.3 øA ³B ' €€€€‚ÿvEBKZ77»€B nC + $€!€Ȁ€€‚ÿP. van Emde Boas, R. Kaas, and E. Zulstra. Design and implementation of an efficient priority queue. Math. Systems Theory, 10:99-127, 1977.1 ³B ŸC ' €€€€‚ÿVeg90ônC ŸD + $€é€Ȁ€€‚ÿG. Vegter. The visibility diagram: a data structure for visibility problems and motion planning. In Proc. second Scand. Workshop on Algorithm Theory (SWAT), volume 447, pages 97-110. Springer Verlag Lecture Notes in Computer Science, 1990.1 ŸC ïD ' €€€€‚ÿVit89‡]ŸD vE * $€º€Ȁ€€‚ÿJ. S. Vitter. Dynamic Huffman coding. ACM Trans. Math. Softw., 15(2):158-167, June 1989.1 ïD §E ' €€€€‚ÿViz64¥uvE LF 0 0€ê€Ȁ€€€€‚ÿV. G. Vizing. On an estimate of the chromatic class of a p-graph (in Russian). Diskret. Analiz, 3:23-30, 1964.1 §E }F ' €€€€‚ÿvL90aÛ°LF XG + $€a€Ȁ€€‚ÿJ. van Leeuwen. Graph algorithms. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science: Algorithms and Complexity, volume A, pages 525-631. MIT Press, 1990.1 }F ‰G ' €€€€‚ÿvL90bŠ|XG /H * $€ø€Ȁ€€‚ÿJ. van Leeuwen, editor. Handbook of Theoretical Computer Science: Algorithms and Complexity, volume A. MIT Press, 1990.0 ‰G _H ' €€€€‚ÿvN63Ï€/H .I + $€I€Ȁ€€‚ÿJ. von Neumann. Various techniques used in connection with random digits. In A. H. Traub, editor, John von Neumann, Collected Works, volume 5. Macmillan, 1963.1 _H _I ' €€€€‚ÿVos92e.I îI * $€Ê€Ȁ€€‚ÿS. Voss. Steiner's problem in graphs: heuristic methods. Discrete Applied Mathematics, 40, 1992.1 _I J ' €€€€‚ÿWar62qGîI J * $€Ž€Ȁ€€‚ÿS. Warshall. A theorem on boolean matrices. J. ACM, 9:11-12, 1962.1 J ÁJ ' €€€€‚ÿWat95~TJ ?K * $€š€Ȁ€€‚ÿM. Waterman. Introduction to Computational Biology. Chapman Hall, London, 1995.2 ÁJ qK ' €€€€‚ÿWBCS77ïÄ?K `L + $€‰€Ȁ€€‚ÿJ. Weglarz, J. Blazewicz, W. Cellary, and R. Slowinski. An automatic revised simplex method for constrained resource network scheduling. ACM Trans. Math. Softw., 3(3):295-300, September 1977.1 qK ‘L ' €€€€‚ÿWei73«€`L \Á âÁ # €6€€‚ÿAbout this document ... |l€Á ^Ä  îé€€†"€!çhxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ„œ4õ€†"€‰€æ…œ4õ€†"€‰€憜4õ€†"€‰€€çhxÊ«€‰€€çbxÊ«€‰€‚€‚€€‚æ‡œ4õ€€€‰€€泜4õ€‰€‚ÿ Up: The Algorithm Design Manual Previous: IndexAbout this document ...The Algorithm Design Manual This document was generated using the LaTeX2HTML translator Version 96.1 (Feb 5, 1996) Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.ŸŠâÁ Å 4 6€€€€€€‚€‚ÿThe command line arguments were: latex2html book.tex.The translation was initiated by Algorithms on Mon Jun 2 23:33:50 EDT 1997)^Ä EÅ & €€€‚ÿS+Å ˜Å ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)EÅ ÁÅ & €€€‚ÿµf˜Å vÆ O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÁÅ §Æ 1Uÿÿÿÿÿÿÿÿ!ÿÿÿÿ§Æ ËÆ $vÆ ËÆ " €€€ ÿ@§Æ Ç 1" ÿÿÿÿÿÿÿÿ" Ç AÇ  Acknowledgments6ËÆ AÇ # €&€€‚ÿAcknowledgments gb Ç šË  ؁Õ€€ç!8ۀ†"€ ‰€çhxÊ«€†"€ ‰€ç_xÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€檜4õ€†"€‰€æ«œ4õ€†"€‰€欜4õ€†"€‰€€ç!8ۀ‰€€çhxÊ«€‰€€ç_xÊ«€‰€‚€‚€‚ÿ Next: Caveat Up: The Algorithm Design Manual Previous: PrefaceAcknowledgmentsI would like to thank several people for their concrete contributions to this project. Ricky Bradley built up the substantial infrastructure required for both the WWW site and CD-ROM in a logical and extensible manner. Zhong Li did a spectacular job drawing most of the catalog figures using xfig and entering the lecture notes that served as the foundation of Part I of this book. Frank Ruscica, Kenneth McNicholas and Dario Vlah all came up big in the pinch, redoing all the audio and helping as the completion deadline approached. Filip Bujanic, David Ecker, David Gerstl, Jim Klosowski, Ted Lin, Kostis Sagonas, Kirsten Starcher, Brian Tria, and Lei Zhao all made contributions at various stages of the project.;AÇ ãÎ ' €)€€‚‚ÿRichard Crandall, Ron Danielson, Takis Metaxas, Dave Miller, Giri Narasimhan, and Joe Zachary all reviewed preliminary versions of the manuscript and/or CD-ROM; their thoughtful feedback helped to shape what you see here. Thanks also to Allan Wylde, the editor of my previous book as well as this one, and Keisha Sherbecoe and Robert Wexler of Springer-Verlag.I learned much of what I know about algorithms along with my graduate students Yaw-Ling Lin, Sundaram Gopalakrishnan, Ting Chen, Francine Evans, Harald Rau, Ricky Bradley, and Dimitris Margaritis. They are the real heroes of many of the war stories related within. Much of the rest I have learned with my Stony Brook friends and colleagues Estie Arkin and Joe Mitchell, who have always been a pleasure to work and be with.tLšË c ( €™€€€‚ÿFinally, I'd like to send personal thanks to several people. Mom, Dad, Len, and Rob all provided moral support. Michael Brochstein took charge of organizing my social life, thus freeing time for me to actually write the book. Through his good ãÎ c ËÆ offices I met Renee. Her love and patience since then have made it all worthwhile.  )ãÎ Œ & €€€‚ÿ­¬c 9  Ёi€€‚ç!8ۀ†"€ ‰€çhxÊ«€†"€ ‰€ç_xÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ­œ4õ€†"€‰€殜4õ€†"€‰€毜4õ€†"€‰€€ç!8ۀ‰€€çhxÊ«€‰€€ç_xÊ«€‰€€‚‚ÿ Next: Caveat Up: The Algorithm Design Manual Previous: Preface AlgorithmsMon Jun 2 23:33:50 EDT 1997)Œ b & €€€‚ÿµf9  O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1b H 1Uÿÿÿÿÿÿÿÿ#ÿÿÿÿH l $ l " €€€ ÿ7H £ 1úÿÿÿÿÿÿÿÿ$£ Ð ™ Caveat- l Ð # €€€‚ÿCaveat Š¡£ v  ؁S€€çaxÊ«€†"€ ‰€çhxÊ«€†"€ ‰€ç*8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ°œ4õ€†"€‰€汜4õ€†"€‰€沜4õ€†"€‰€€çaxÊ«€‰€€çhxÊ«€‰€€ç*8ۀ‰€‚€‚€‚ÿ Next: Contents Up: The Algorithm Design Manual Previous: AcknowledgmentsCaveatIt is traditional for the author to magnanimously accept the blame for whatever deficiencies remain. I don't. Any errors, deficiencies, or problems in this book are somebody else's fault, but I would appreciate knowing about them so as to determine who is to blame.ɘÐ ? 1 0€3€€†"€€‚ÿSteven S. Skiena Department of Computer Science State University of New York Stony Brook, NY 11794-4400 http://www.cs.sunysb.edu/ skiena May 1997)v h & €€€‚ÿS+? » ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)h ä & €€€‚ÿµf» ™ O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ä Ê 1Uÿÿÿÿÿÿÿÿ%ÿÿÿÿÊ î $™ î " €€€ ÿKÊ 9 1àÿÿÿÿÿÿÿÿ&9 z BB Correctness and EfficiencyAî z # €<€€‚ÿCorrectness and Efficiency 9 —  ü€€ç#8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çfxÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÞœ4õ€†"€‰€æÕœ4õ€†"€‰€æÖœ4õ€†"€‰€€ç#8ۀ‰€€çfxÊ«€‰€€çfxÊ«€‰€‚€‚€€€€€€€‚ÿ Next: Correctness Up: Introduction to Algorithms Previous: Introduction to AlgorithmsCorrectness and EfficiencyThroughout this book we will seek algorithms that are correct and efficient, while being easy to implement. All three goals are obviously desirable, but they may not be simultaneously achievable. For this reason, one or more of them are often ignored. Theoretical algorithm designers have traditionally been unconcerned with implementation complexity, since they often do not program their algorithms. Instead, theoreticians focus on efficiency and correctness. Conversely, quick-and-dirty is typically the rule of thumb in industrial settings. Any program that seems to give good enough answers without slowing the application down is acceptable, regardless of whether a better algorithm exists. The issue of finding the best possible answer or achieving maximum efficiency usually does not arise in industry until serious troubles do.   Z&z @ 4 6€M€€€€€€€‚ÿHere, we stress the importance of recognizing the difference between algorithms, which always produce a correct result, and heuristics, which often do a good job without providing any guarantee. We also emphasize the potential efficiency gains resulting from using faster algorithms.  — @ î )— 5@ & €€€‚ÿ& @ [@ # €€€‚ÿg'5@ Â@ @ P€N€ȀçV¹€‰€‚€緖¿c€‰€‚ÿu Correctness u Efficiency&[@ è@ # €€€‚ÿ)Â@ A & €€€‚ÿS+è@ dA ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)A A & €€€‚ÿµfdA BB O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1A sB 1Uÿÿÿÿÿÿÿÿ'ÿÿÿÿsB —B $BB —B " €€€ ÿ< sB ÓB 1,.ÿÿÿÿÿÿÿÿ(ÓB C ~Ï Correctness2—B C # €€€‚ÿCorrectness ÓB G  ô€€ç$8ۀ†"€ ‰€ç"8ۀ†"€ ‰€ç"8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æל4õ€†"€‰€æØœ4õ€†"€‰€æÙœ4õ€†"€‰€€ç$8ۀ‰€€ç"8ۀ‰€€ç"8ۀ‰€‚€ ‚€‚€€€€‚ÿ Next: Efficiency Up: Correctness and Efficiency Previous: Correctness and EfficiencyCorrectness It is seldom obvious whether a given algorithm correctly solves a given problem. This is why correct algorithms usually come with a proof of correctness, which is an explanation of why we know that the algorithm correctly takes all instances of the problem to the desired result. In this book, we will not stress formal proofs of correctness, primarily because they take substantial mathematical maturity to properly appreciate and construct. However, before we go further it is important to demonstrate why ``it's obvious'' never suffices as a proof of correctness and usually is flat-out wrong.  T/C rJ % €_€€‚ÿTo illustrate, let us consider a problem that often arises in manufacturing, transportation, and testing applications. Suppose we are given a robot arm equipped with a tool, say a soldering iron. In manufacturing circuit boards, all the chips and other components must be fastened onto the substrate. More specifically, each chip has a set of contact points (or wires) that must be soldered to the board. To program the robot arm to perform this soldering job, we must first construct an ordering of the contact points, so that the robot visits (and solders) the first contact point, then visits the second point, third, and so forth until the job is done. The robot arm must then proceed back to the first contact point to prepare for the next board, thus turning the tool-path into a closed tour, or cycle.   Å{G 7M J b€÷€€€‚€€€€€€‚€€€€‚ÿSince robots are expensive devices, we want to find the tour that minimizes the time it takes to assemble the circuit board. A reasonable assumption is that the robot arm moves with fixed speed, so that the time it takes to travel between two points is the same as its distance. In short, we must solve the following algorithm problem:   Input: A set S of n points in the plane. Output: What is the shortest cycle tour that visits each point in the set S? You are given the job of programming the robot arm. Stop right now and think about an algorithm to solve this problem. I'll be happy to wait until you find one.)rJ `M & €€€‚ÿ?ä7M ŸO [ „€Ó€€‚€€€†"€"†"€#†"€#†"€"†"€"‚ÿSeveral possible algorithms might come to mind to solve this problem. Perhaps the most popular idea is the nearest-neighbor heuristic. Starting from some point , we walk first to its nearest neighbor . From , we walk to its nearest unvisited neighbor, thus excluding only as a candidate. We now repeat this process until we run out of unvisited points, at which time we return to to close off the tour. In pseudocode, the nearest-neighbor heuristic looks like this:  zF`M %€ 4 8€Ž€€†"€$€€‚ÿ    Figure: A good example for the neaŸO %€ —B rest neighbor heuristic­ŸO ҁ § €€‚€€‚†"€"€€‚†"€%‚€€‚‚€€€€‚†"€&†"€'‚†"€&‚†"€"†"€(‚ÿ NearestNeighborTSP(P) Pick and visit an initial point from P i = 0 While there are still unvisited points i = i+1 Select to be the closest unvisited point to Visit Return to from Tÿ%€ &„ U x€€€çT¹€†"€‰€†"€)†"€&†"€*‚ÿThis algorithm has a lot to recommend it. It is simple to understand and implement. It makes sense to visit nearby points before we visit faraway points if we want to minimize the total travel time. The algorithm works perfectly on the example in Figure . The nearest neighbor rule is very efficient, for it looks at each pair of points at most twice, once when adding to the tour, the other when adding . Against all these positives there is only one problem. This algorithm is completely wrong.è–ҁ ˆ R r€1€€†"€+€€€‚€çùdˀ†"€‰€‚ÿ    Figure: A bad example for the nearest neighbor heuristic Wrong? How can it be wrong? The algorithm always finds a tour, but the trouble is that it doesn't necessarily find the shortest possible tour. It doesn't necessarily even come close. Consider the set of points in Figure , all of which lie spaced along a line. The numbers describe the distance that each point lies to the left or right of the point labeled `0'. When we start from the point `0' and repeatedly walk to the nearest unvisited neighbor, you will see that we keep jumping left-right-left-right over `0'. A much better (indeed optimal) tour for these points starts from the leftmost point and visits each point as we walk right before returning at the rightmost point. Try now to imagine your boss's delight as she watches a demo of your robot arm hopscotching left-right-left-right during the assembly of such a simple board.  ×&„ ‹ 0 .€±€€†"€"‚‚ÿ``But wait,'' you might be saying. ``The problem was in starting at point `0'. Instead, why don't we always start the nearest-neighbor rule using the leftmost point as the starting point ? By doing this, we will find the optimal solution on this example.''That is 100% true, at least until we rotate our example 90 degrees. Now all points are equally leftmost. If the point `0' were moved just slightly to the left, it would be picked as the starting point. Now the robot arm will hopscotch up-down-up-down instead of left-right-left-right, but the travel time will be just as bad as before. No matter what you do to pick the first point, the nearest neighbor rule is doomed not to work correctly on certain point sets. uˆ µ + $€ë€€€€‚ÿMaybe what we need is a different approach. Always walking to the closest point is too restrictive, since it seems to trap us into making moves we didn't want. A different idea would be to repeatedly connect the closest pair of points whose connection will not cause a cycle or a three-way branch to be formed. Eventually, we will end up with a single chain containing all the points in it. At any moment during the execution of this closest-pair heuristic, we will have a set of partial paths to merge, where each vertex begins as its own partial path. Connecting the final two endpoints gives us a cycle. In pseudocode:  £Ó‹ dÁ Ð n³€€‚€€‚€€€€‚†"€,‚€€€€‚€€‚†"€-†"€.†"€/€€€€€€‚†"€0‚‚€çùdˀ†"€‰€€€€€‚ÿ ClosestPairTSP(P) Let n be the number of points in set P. For i=1 to n-1 do For each pair of endpoints (,t) of distinct partial paths if then , , and d = dist(,t) Connect by an edge Connect the two endpoints by an edge This closest-pair rule does the right thing on the example in Figure . It sµ dÁ —B tarts by connecting `0' to its immediate neighbors, the points 1 and -1. Subsequently, the next closest pair will alternate left-right, growing the central path by one link at a time. The closest-pair heuristic is somewhat more complicated and less efficient than the previous one, but at least it gives the right answer. On this example.sôµ ×Ä  ̀󀀁†"€1€€‚çódˀ†"€‰€€€€€†"€2çódˀ†"€‰€†"€3‚ÿ  Figure: A bad example for the closest-pair heuristic.  But not all examples. Consider what the algorithm does on the point set in Figure (a). It consists of two rows of equally spaced points, with the rows slightly closer together (distance 1-e) than the neighboring points are spaced within each row (distance 1+e). Thus the closest pairs of points stretch across the gap, not around the boundary. After we pair off these points, the closest remaining pairs will connect these pairs alternately around the boundary. The total path length of the closest-pair tour is . Compared to the tour shown in Figure (b), we travel over 20% farther than necessary when . Examples exist where the penalty is considerably worse than this.óÆdÁ ÊÆ - (€€€‚€€‚ÿThus this second algorithm is also wrong. Which one of these algorithms performs better? You can't tell just by looking at them. Clearly, both heuristics can end up with very bad tours on very innocent-looking input.  At this point, you might wonder what a correct algorithm for the problem looks like. Well, we could try all enumerating all possible orderings of the set of points, and then select the ordering that minimizes the total length:  YÎ×Ä #Ê ‹ 䀫€€‚‚†"€,‚€€†"€4€€‚†"€5†"€6†"€7‚†"€8‚€†"€9‚ÿ OptimalTSP(P) For each of the n! permutations of points P If then and Return Since all possible orderings are considered, we are guaranteed to end up with the shortest possible tour. This algorithm is correct, since we pick the best of all the possibilities. The problem is that it is also extremely slow. The fastest computer in the world couldn't hope to enumerate all 20! = 2,432,902,008,176,640,000 orderings of 20 points within a day. For real circuit boards, where , forget about it. All of the world's computers working full time wouldn't come close to finishing your problem before the end of the universe, at which point it presumably becomes moot.‘LÊÆ ŽÌ E X€›€€€€çÃß{+€†"€‰€‚€‚ÿThe quest for an efficient algorithm to solve this so-called traveling salesman problem will take us through much of this book. If you need to know how the story ends, check out the catalog entry for the traveling salesman problem in Section .   Hopefully, this example has opened your eyes to some of the subtleties of algorithm correctness. Ensuring the optimal answer on all possible inputs is a difficult but often achievable goal. Seeking counterexamples that break pretender algorithms is a tricky business, but it is an important part of the algorithm design process.)#Ê ÝÌ & €€€‚ÿÃÂŽÌ  Î  Ё•€€‚ç$8ۀ†"€ ‰€ç"8ۀ†"€ ‰€ç"8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÚœ4õ€†"€‰€æÛœ4õ€†"€‰€æÜœ4õ€†"€‰€€ç$8ۀ‰€€ç"8ۀ‰€€ç"8ۀ‰€€‚‚ÿ Next: Efficiency Up: Correctness and Efficiency Previous: Correctness and Efficiency AlgorithmsMon Jun 2 23:33:50 EDT 1997)ÝÌ ÉÎ & €€€‚ÿµf Î ~Ï O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÉÎ ¯Ï 1Uÿÿÿÿÿÿÿÿ)ÿÿÿÿ¯Ï $~Ï " €€€ ÿ¯Ï ~Ï ; ¯Ï G1õ ÿÿÿÿÿÿÿÿ*Gx< Efficiency1 x# €€€‚ÿEfficiency kQGã ‚µ€€ç%8ۀ†"€ ‰€ç"8ۀ†"€ ‰€ç#8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÝœ4õ€†"€‰€挪Û/€†"€‰€惪Û/€†"€‰€€ç%8ۀ‰€€ç"8ۀ‰€€ç#8ۀ‰€‚€ ‚€çlªù(€†"€‰€‚ÿ Next: Expressing Algorithms Up: Correctness and Efficiency Previous: CorrectnessEfficiencyThe skilled algorithm designer is an efficiency expert. Through cunning and experience we seek to get jobs done as quickly as possible. The benefits of finding an efficient algorithm for a computationally expensive job can be mind-boggling, as illustrated by several of the war stories scattered through this book. Section shows how clever algorithms yielded a program that was 30,000 times faster than our initial attempt!Ê—x­3 4€/€€‚€€€€‚ÿUpon realizing that a given application runs too slowly, the practical person usually asks the boss to buy them a faster machine. Sometimes this is the cheapest option. However, the potential win from faster hardware is typically limited to a factor of ten or so, due to technical constraints (they don't make faster machines) or economic ones (the boss won't pay for it).  To realize larger performance improvements, we must seek better algorithms. As we will show, a faster algorithm running on a slower computer will always win for sufficiently large instances. Always. Usually, problems don't have to get very large before the faster algorithm wins.É¡ãv ( €C€€€‚ÿBe aware that there are situations where finding the most efficient algorithm for a job is a complete waste of programmer effort. In any program, there is usually one bottleneck that takes the majority of computing time. The typical claim is that 90% of the run time of any program is spent in 10% of the code. Optimizing the other 90% of the program will have little impact on the total run time. Further, many programs are written with only one or two special instances in mind. If the program will be run just a few times, or if the job can easily be run overnight, it probably does not pay to make the programmer work harder in order to reduce cycle consumption.  )­Ÿ & €€€‚ÿ¿Ÿv ^  Ё€€‚ç%8ۀ†"€ ‰€ç"8ۀ†"€ ‰€ç#8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€愪Û/€†"€‰€慪Û/€†"€‰€憪Û/€†"€‰€€ç%8ۀ‰€€ç"8ۀ‰€€ç#8ۀ‰€€‚‚ÿ Next: Expressing Algorithms Up: Correctness and Efficiency Previous: Correctness AlgorithmsMon Jun 2 23:33:50 EDT 1997)Ÿ ‡ & €€€‚ÿµf^ < O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1‡ m 1Uÿÿÿÿÿÿÿÿ+ÿÿÿÿm ‘ $< ‘ " €€€ ÿFm × 1Ÿ ÿÿÿÿÿÿÿÿ,×  –IExpressing Algorithms<‘  # €2€€‚ÿExpressing Algorithms A<× `A ؁‰€€ç&8ۀ†"€ ‰€çfxÊ«€†"€ ‰€ç$8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€懪Û/€†"€‰€截Û/€†"€‰€扪Û/€†"€‰€€ç&8ۀ‰€€çfxÊ«€‰€€ç$8ۀ‰€‚€‚€‚ÿ Next: Keeping Score Up: Introduction to Algorithms Previous: EfficiencyExpressing AlgorithmsDescribing algorithms requires a notation for expressing a sequence of steps to be performed. The three most common options are (1) English, (2) pseudocode, or (3) a real programming language. Pseudocode is perhaps the most mysterious of the bunch, but it is best defined as a programming language that never complains about `A‘  syntax errors. All three methods can be useful in certain circumstances, since there is a natural tradeoff between greater ease of expression and precision. English is the most natural but least precise language, while C and Pascal are precise but difficult to write and understand. Pseudocode is useful because it is a happy medium.    &û †C+ $€÷€€€€‚ÿThe correct choice of which notation is best depends upon which of the three methods you are most comfortable with. I prefer to describe the ideas of an algorithm in English, moving onto a more formal, programming-language-like pseudocode to clarify sufficiently tricky details of the algorithm. A common mistake among my students is to use pseudocode to take an ill-defined idea and dress it up so that it looks more formal. In the real world, you only fool yourself when you pull this kind of stunt.S`AÙFC T€#€€€€ç%ɲ+€†"€‰€€‚ÿThe implementation complexity of an algorithm is usually why the fastest algorithm known for a problem may not be the most appropriate for a given application. An algorithm's implementation complexity is often a function of how it has been described.   Fast algorithms often make use of very complicated data structures, or use other complicated algorithms as subroutines. Turning these algorithms into programs requires building implementations of every substructure. Each catalog entry in Section points out available implementations of algorithms to solve the given problem. Hopefully, these can be used as building blocks to reduce the implementation complexity of algorithms that use them, thus making more complicated algorithms worthy of consideration in practice.)†CG& €€€‚ÿ¶µÙFžH Ё{€€‚ç&8ۀ†"€ ‰€çfxÊ«€†"€ ‰€ç$8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€抪Û/€†"€‰€拪Û/€†"€‰€æ ©Û/€†"€‰€€ç&8ۀ‰€€çfxÊ«€‰€€ç$8ۀ‰€€‚‚ÿ Next: Keeping Score Up: Introduction to Algorithms Previous: Efficiency AlgorithmsMon Jun 2 23:33:50 EDT 1997)GáH& €€€‚ÿµfžH–IO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1áHÇI1Uÿÿÿÿÿÿÿÿ-ÿÿÿÿÇIëI$–IëI" €€€ ÿ> ÇI)J19ÿÿÿÿÿÿÿÿ.)J]JfKeeping Score4ëI]J# €"€€‚ÿKeeping Score 0%)JM  ä[€€ç'8ۀ†"€ ‰€çfxÊ«€†"€ ‰€ç%8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ©Û/€†"€‰€æ©Û/€†"€‰€æ©Û/€†"€‰€€ç'8ۀ‰€€çfxÊ«€‰€€ç%8ۀ‰€‚€‚€€€‚ÿ Next: The RAM Model of Up: Introduction to Algorithms Previous: Expressing AlgorithmsKeeping ScoreAlgorithms are the most important, durable, and original part of computer science because they can be studied in a language- and machine-independent way. This means that we need techniques that enable us to compare algorithms without implementing them. Our two most important tools are (1) the RAM model of computation and (2) asymptotic analysis of worst-case complexity.   ^6]JëN( €m€€€‚ÿThis method of keeping score will be the most mathematically demanding part of this book. However, it is important to understand why we need both of these tools to analyze the performance of algorithms. Once you understand the intuition behind these ideas, the formalism becomes a lot easier to deal with.)MO& €€€‚ÿ&ëN:O# €€€‚ÿ–VOÐO@ P€¬€ȀçÚAӀ€‰€‚€ç;ÐÙ߀‰€‚ÿu The RAM Model of Computation u Best, Worst, and Average-Case Complexity&:O €# €€€‚ÿÐO €ëI)ÐO5€& €€€‚ÿS+ €ˆ€( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)5€±€& €€€‚ÿµfˆ€fO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1±€—1Uÿÿÿÿÿÿÿÿ/ÿÿÿÿ—»$f»" €€€ ÿM—‚1œÿÿÿÿÿÿÿÿ0‚K‚fÂThe RAM Model of ComputationC »K‚# €@€€‚ÿThe RAM Model of Computation |q‚DŽ  äó€€ç(8ۀ†"€ ‰€ç&8ۀ†"€ ‰€ç&8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ©Û/€†"€‰€æ©Û/€†"€‰€æ©Û/€†"€‰€€ç(8ۀ‰€€ç&8ۀ‰€€ç&8ۀ‰€‚€ ‚€€€‚ÿ Next: BestWorst, and Average-Case Up: Keeping Score Previous: Keeping ScoreThe RAM Model of ComputationMachine-independent algorithm design depends upon a hypothetical computer called the Random Access Machine or RAM. Under this model of computation, we are confronted with a computer where:  0÷K‚÷†9 @€ï€Ȁ€€‚€€€€‚ÿu Each ``simple'' operation (+, *, -, =, if, call) takes exactly 1 time step.u Loops and subroutines are not considered simple operations. Instead, they are the composition of many single-step operations. It makes no sense for ``sort'' to be a single-step operation, since sorting 1,000,000 items will take much longer than sorting 10 items. The time it takes to run through a loop or execute a subprogram depends upon the number of loop iterations or the specific nature of the subprogram.  òÊDŽé‡( €•€Ȁ€‚ÿu Each memory access takes exactly one time step, and we have as much memory as we need. The RAM model takes no notice of whether an item is in cache or on the disk, which simplifies the analysis.  "û÷† Œ' €÷€€‚‚ÿUnder the RAM model, we measure the run time of an algorithm by counting up the number of steps it takes on a given problem instance. By assuming that our RAM executes a given number of steps per second, the operation count converts easily to the actual run time.The RAM is a simple model of how computers perform. A common complaint is that it is too simple, that these assumptions make the conclusions and analysis too coarse to believe in practice. For example, multiplying two numbers takes more time than adding two numbers on most processors, which violates the first assumption of the model. Memory access times differ greatly depending on whether data sits in cache or on the disk, thus violating the third assumption. Despite these complaints, the RAM is an excellent model for understanding how an algorithm will perform on a real computer. It strikes a fine balance by capturing the essential behavior of computers while being simple to work with. We use the RAM model because it is useful in practice.Žd釙* "€É€€‚€‚ÿEvery model has a size range over which it is useful. Take, for example, the model that the earth is flat.   You might argue that this is a bad model, since the earth is not flat. However, when laying the foundation of a house, the flat earth model is sufficiently accurate that it can be reliably used. Further, it is so much easier to manipulate a flat-earth model that it is inconceivable that you would try to think spherically when you don't have to.The same situation is true with the RAM model of computation. We make an abstraction that in general is very useful. It is quite difficult to design an algorithm such that the RAM model will give you substantially misleading results, by performing either much better or much worse in practice than the model suggests. The robustness of the RAM enables us to analyze algorithms in a machine-independent way.) Œ& €€€‚ÿº¹™ˆÁ Ёƒ€€‚ç(8ۀ†"€ ‰€ç&8ۀˆÁ»†"€ ‰€ç&8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ©Û/€†"€‰€æ©Û/€†"€‰€æ©Û/€†"€‰€€ç(8ۀ‰€€ç&8ۀ‰€€ç&8ۀ‰€€‚‚ÿ Next: BestWorst, and Average-Case Up: Keeping Score Previous: Keeping Score AlgorithmsMon Jun 2 23:33:50 EDT 1997)±Á& €€€‚ÿµfˆÁfÂO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1±Á—Â1Uÿÿÿÿÿÿÿÿ1ÿÿÿÿ—»Â$f»Â" €€€ ÿY(—ÂÃ1áÿÿÿÿÿÿÿÿ2ÃcÃþBest, Worst, and Average-Case ComplexityO,»ÂcÃ# €X€€‚ÿBest, Worst, and Average-Case Complexity ÐÅÃ3Æ  䁛€€ç)8ۀ†"€ ‰€ç&8ۀ†"€ ‰€ç'8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ4©Û/€†"€‰€æ+©Û/€†"€‰€æ,©Û/€†"€‰€€ç)8ۀ‰€€ç&8ۀ‰€€ç'8ۀ‰€‚€ ‚€€€‚ÿ Next: The Big Oh Notation Up: Keeping Score Previous: The RAM Model ofBest, Worst, and Average-Case ComplexityUsing the RAM model of computation, we can count how many steps our algorithm will take on any given input instance by simply executing it on the given input. However, to really understand how good or bad an algorithm is, we must know how it works over all instances.X'cËÉ1 0€O€€€€€€‚ÿTo understand the notions of the best, worst, and average-case complexity, one must think about running an algorithm on all possible instances of data that can be fed to it. For the problem of sorting, the set of possible input instances consists of all the possible arrangements of all the possible numbers of keys. We can represent every input instance as a point on a graph, where the x-axis is the size of the problem (for sorting, the number of items to sort) and the y-axis is the number of steps taken by the algorithm on this instance. Here we assume, quite reasonably, that it doesn't matter what the values of the keys are, just how many of them there are and how they are ordered. It should not take longer to sort 1,000 English names than it does to sort 1,000 French names, for example.©]3Æ4ËL f€¿€€†"€:€€‚ç3ÐÙ߀†"€‰€‚ÿ  Figure: Best, worst, and average-case complexity  As shown in Figure , these points naturally align themselves into columns, because only integers represent possible input sizes. After all, it makes no sense to ask how long it takes to sort 10.57 items. Once we have these points, we can define three different functions over them:¥G‹ÉÙÍ^ Š€€Ȁ€€€€€€‚€€€€€€‚€€€€€€‚ÿu The worst-case complexity of the algorithm is the function defined by the maximum number of steps taken on any instance of size n. It represents the curve passing through the highest point of each column.  u The best-case complexity of the algorithm is the function defined by the minimum number of steps taken on any instance of size n. It represents the curve passing through the lowest point of each column.  u Finally, the average-case complexity of the algorithm is the function defined by the average number of steps taken on any instance of size n.  ‚K4Ëg7 <€—€€€€€€€€‚ÿIn practice, the most useful of these three measures proves to be the worst-case complexity, which many people find counterintuitive. To illustrate why worst-case analysis is important, consider trying to project what will happen to you if you bring n dollars to gamble in a casino.   The best case, that you walk out owning the place, is possible but so unlikely that you should place no credence in it. The worst case, that you lose all n dollars, is easy to calculate and distressingly lÙÍg»Âikely to happen. The average case, that the typical person loses 87.32% of the money that they bring to the casino, is difficult to compute and its meaning is subject to debate. What exactly does average mean? Stupid people lose more than smart people, so are you smarter or dumber than the average person, and by how much? People who play craps lose more money than those playing the nickel slots. Card counters at blackjack do better on average than customers who accept three or more free drinks. We avoid all these complexities and obtain a very useful result by just considering the worst case.ÛªÙÍB1 0€W€€†"€;€‚ÿThe important thing to realize is that each of these time complexities defines a numerical function, representing time versus problem size. These functions are as well-defined as any other numerical function, be it or the price of General Motors stock as a function of time. Time complexities are complicated functions, however. In order to simplify our work with such messy functions, we will need the big Oh notation.)gk& €€€‚ÿµŽB  Ёy€€‚ç)8ۀ†"€ ‰€ç&8ۀ†"€ ‰€ç'8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ-©Û/€†"€‰€æ.©Û/€†"€‰€æ/©Û/€†"€‰€€ç)8ۀ‰€€ç&8ۀ‰€€ç'8ۀ‰€€‚‚ÿ Next: The Big Oh Notation Up: Keeping Score Previous: The RAM Model of AlgorithmsMon Jun 2 23:33:50 EDT 1997)kI& €€€‚ÿµf þO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1I/1Uÿÿÿÿÿÿÿÿ3ÿÿÿÿ/S$þS" €€€ ÿD/—1Îÿÿÿÿÿÿÿÿ4—щIThe Big Oh Notation:SÑ# €.€€‚ÿThe Big Oh Notation ›—l  ‚€€çU8ۀ†"€ ‰€çfxÊ«€†"€ ‰€ç(8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ0©Û/€†"€‰€æ1©Û/€†"€‰€æ2©Û/€†"€‰€€çU8ۀ‰€€çfxÊ«€‰€€ç(8ۀ‰€‚€‚€ç–^à>€†"€‰€‚ÿ Next: Growth Rates Up: Introduction to Algorithms Previous: BestWorst, and Average-CaseThe Big Oh NotationWe have agreed that the best, worst, and average-case complexities of a given algorithm are numerical functions of the size of the instances. However, it is difficult to work with these functions exactly because they are often very complicated, with many little up and down bumps, as shown in Figure . Thus it is usually cleaner and easier to talk about upper and lower bounds of such functions. This is where the big Oh notation comes into the picture.   g0ÑÓ 7 <€c€€†"€<€€‚‚ÿ  Figure: Upper and lower bounds smooth out the behavior of complex functions  We seek this smoothing in order to ignore levels of detail that do not impact our comparison of algorithms. Since running our algorithm on a machine that is twice as fast will cut the running times of all algorithms by a multiplicative constant of two, such constant factors would be washed away in upgrading machines. On the RAM we ignore such constant factors anyway and therefore might as well ignore them when comparing two algorithms. We use the following notation:X l 7AK d‚3€Ȁ€€€€€€€€€€€†"€=€€€€€€€€€€€€†"€>€€‚€€†"€?†"€=€€€€€€€€€€€€†"€@€€‚€€†"€A†"€B€€€€†"€C€€€€€€†"€D†"€E†"€F†"€G€€€€€€€€‚ÿu f(n) = O(g(n)) means is an upper bound on f(n). Thus there exists some constant c such that f(n) is always , for large enough n.  u means is a lower bound on f(n). Thus there existÓ 7ASs some constant c such that f(n) is always , for large enough n.  u means is an upper bound on f(n) and is a lower bound on f(n), for large enough n. Thus there exists constants and such that and . This means that g(n) is a nice, tight bound on f(n).  Ÿ#Ó ÖC| ƀS€€†"€H€€€€†"€I†"€J‚ç›^à>€†"€‰€†"€K€€†"€K‚ÿ  Figure: Illustrating the (a) O, (b) , and (c) notations  Got it? These definitions are illustrated in Figure . All of these definitions imply a constant beyond which they are always satisfied. We are not concerned about small values of n, i.e. anything to the left of . After all, do you really care whether one sorting algorithm sorts six items faster than another algorithm, or which one is faster when sorting 1,000 or 1,000,000 items? The big Oh notation enables us to ignore details and focus on the big picture.æz7AŒFl Š€€€‚†"€L‚†"€M‚†"€N‚†"€O€€†"€P†"€Q€‚ÿMake sure you understand this notation by working through the following examples. We choose certain constants in the explanations because they work and make a point, but you are free to choose any constant that maintains the same inequality:The big Oh notation provides for a rough notion of equality when comparing functions. It is somewhat jarring to see an expression like , but its meaning can always be resolved by going back to the definitions in terms of upper and lower bounds. It is perhaps most instructive to read `=' as meaning one of the functions that are. Clearly, is one of functions that are .)ÖCåF& €€€‚ÿÆÅŒF«H Ё›€€‚çU8ۀ†"€ ‰€çfxÊ«€†"€ ‰€ç(8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ3©Û/€†"€‰€æ_©Û/€†"€‰€æV©Û/€†"€‰€€çU8ۀ‰€€çfxÊ«€‰€€ç(8ۀ‰€€‚‚ÿ Next: Growth Rates Up: Introduction to Algorithms Previous: BestWorst, and Average-Case AlgorithmsMon Jun 2 23:33:50 EDT 1997)åFÔH& €€€‚ÿµf«H‰IO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÔHºI1Uÿÿÿÿÿÿÿÿ5ÿÿÿÿºIÞI$‰IÞI" €€€ ÿ= ºIJ1ëÿÿÿÿÿÿÿÿ6JNJŠ‡Growth Rates3ÞINJ# € €€‚ÿGrowth Rates ¹EJOt ¶‚¥€€çL8ۀ†"€ ‰€çfxÊ«€†"€ ‰€ç)8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æW©Û/€†"€‰€æX©Û/€†"€‰€æY©Û/€†"€‰€€çL8ۀ‰€€çfxÊ«€‰€€ç)8ۀ‰€‚€‚€†"€R†"€S€€€€€€€€€€çBה€†"€‰€çBה€†"€‰€€€€€†"€T‚ÿ Next: Logarithms Up: Introduction to Algorithms Previous: The Big Oh NotationGrowth RatesIn working with the big Oh notation, we cavalierly discard the multiplicative constants. The functions and are treated identically, even though g(n) is a million times larger than f(n) for all values of n. The method behind this madness is illustrated by Figure , which tabulates the growth rate of several functions arising in algorithm analysis, on problem instances of reasonable size. Specifically, Figure shows how long it takes for algorithms that use f(n) operations to complete on a fast computer where each operation takes one nanosecond ( seconds). Study the table for a few minutes and the following conclusions become apparent:    vKNJ}O+ &€–€€€€‚ÿ  Figure:   Growth rates of common functions measured in nanoseconds=ÆOƁw Œ€‘€Ȁ€€€€‚€€€€€€‚€€†"€U€€‚€€†"€P€€€€‚ÿu All of}OƁÞI these algorithms take about the same amount of time for n=10.u The algorithm whose running time is n! becomes useless well before n=20.u The algorithm whose running time is has a greater operating range, but it becomes impractical for n > 40.u The algorithm whose running time is is perfectly reasonable up to about n=100, but it quickly deteriorates with larger inputs. For n > 1,000,000 it likely to be hopeless.Ÿ}O˂G \€€Ȁ€€€†"€V‚€€†"€W‚ÿu Both the n and algorithms remain practical on inputs of up to one billion items.u You can't hope to find a real problem where an algorithm is going to be too slow in practice.ü¹ƁDŽC T€y€€†"€X†"€Y†"€Z€‚ÿThe bottom line is that even by ignoring constant factors, we can get an excellent idea of whether a given algorithm will be able to run in a reasonable amount of time on a problem of a given size. An algorithm whose running time is seconds will beat one whose running time is seconds only so long as . Such enormous constant factor differences between algorithms occur in practice far less frequently than such large problems do.)˂ð„& €€€‚ÿŒ»Ç„¬† Ё‡€€‚çL8ۀ†"€ ‰€çfxÊ«€†"€ ‰€ç)8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æZ©Û/€†"€‰€æ[©Û/€†"€‰€æ\©Û/€†"€‰€€çL8ۀ‰€€çfxÊ«€‰€€ç)8ۀ‰€€‚‚ÿ Next: Logarithms Up: Introduction to Algorithms Previous: The Big Oh Notation AlgorithmsMon Jun 2 23:33:50 EDT 1997)ð„Ն& €€€‚ÿµf¬†Š‡O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Ն»‡1Uÿÿÿÿÿÿÿÿ7ÿÿÿÿ»‡߇$Š‡߇" €€€ ÿ; »‡ˆ1oÿÿÿÿÿÿÿÿ8ˆKˆ•Logarithms1߇Kˆ# €€€‚ÿLogarithms £hˆîŒ; D‚退çM8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çU8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ]©Û/€†"€‰€æ^©Û/€†"€‰€抩Û/€†"€‰€€çM8ۀ‰€€çfxÊ«€‰€€çU8ۀ‰€‚€‚€€€†"€[†"€\†"€WçBה€†"€‰€‚ÿ Next: Modeling the Problem Up: Introduction to Algorithms Previous: Growth RatesLogarithmsLogarithm is an anagram of algorithm, but that's not why the wily algorist needs to know what logarithms are and where they come from. You've seen the button on your calculator but have likely forgotten why it is there. A logarithm is simply an inverse exponential function. Saying is equivalent to saying that . Exponential functions are functions that grow at a distressingly fast rate, as anyone who has ever tried to pay off a mortgage or bank loan understands. Thus inverse exponential functions, i.e. logarithms, grow refreshingly slowly. If you have an algorithm that runs in time, take it and run. As shown by Figure , this will be blindingly fast even on very large problem instances.  ÓxKˆÍÀ[ „€õ€€†"€W€€€€€€€€€€€€†"€]‚ÿBinary search is an example of an algorithm that takes time. In a telephone book with n names, you start by comparing the name that you are looking for with the middle, or (n/2)nd name, say Monroe, Marilyn. Regardless of whether you are looking someone before this middle name (Dean, James) or after it (Presley, Elvis), after only one such comparison you can forever disregard one half of all the names in the book. The number of steps the algorithm takes equals the number of times we can halve n before only one name is left. By definition, this is exactly . Thus twenty comparisons suffice to find any name in the million-name Manhattan phone book! The power of binary searîŒÍÀ߇ch and logarithms is one of the most fundamental ideas in the analysis of algorithms. This power becomes apparent if we could imagine living in a world with only unsorted telephone books.    %ëîŒòÂ: B€Ù€€çÛÕÅ<€†"€‰€‚ÿFigure is another example of logarithms in action. This table appears in the Federal Sentencing Guidelines, used by courts throughout the United States. These guidelines are an attempt to standardize criminal sentences, so that a felon convicted of a crime before one judge receives the same sentence as they would before a different judge. To accomplish this, the judges have prepared an intricate point function to score the depravity of each crime and map it to time-to-serve.  ÏwÍÀÁÆX ~€ó€€†"€^€€‚çÛÕÅ<€†"€‰€€€€€‚ÿ  Figure:   The Federal Sentencing Guidelines for FraudFigure gives the actual point function for fraud, a table mapping dollars stolen to points. Notice that the punishment increases by one level each time the amount of money stolen roughly doubles. That means that the level of punishment (which maps roughly linearly to the amount of time served) grows logarithmically with the amount of money stolen. Think for a moment about the consequences of this. Michael Milken sure did. It means that the total sentence grows extremely slowly with the amount of money you steal. Knocking off five liquor stores for $10,000 each will get you far more time than embezzling $100,000 once. The corresponding benefit of stealing really large amounts of money is even greater. The moral of logarithmic growth is clear: ``If you are gonna do the crime, make it worth the time!'' nJòÂ/Ç$ €”€€‚ÿTwo mathematical properties of logarithms are important to understand:‡=ÁƶÈJ b€€Ȁ€€€†"€_†"€`†"€a‚ÿu The base of the logarithm has relatively little impact on the growth rate: Compare the following three values: , , and . A big change in the base of the logarithm produces relatively little difference in the value of the log. This is a consequence of the formula for changing the base of the logarithm:3/ÇéÈ. ,€ €Ȁ†"€b‚ÿ\¶ÈEÊG \€-€Ȁ€€€€†"€c€€€€‚ÿChanging the base of the log from a to c involves multiplying or dividing by . This will be lost to the big Oh notation whenever a and c are constants, as is typical. Thus we are usually justified in ignoring the base of the logarithm when analyzing algorithms.  ËéÈHÌ8 >€›€Ȁ†"€d†"€e‚ÿWhen performing binary search in a telephone book, how important is it that each query split the book exactly in half? Not much. Suppose we did such a sloppy job of picking our queries such that each time we split the book 1/3 to 2/3 instead of 1/2 to 1/2. For the Manhattan telephone book, we now use queries in the worst case, not a significant change from . The power of binary search comes from its logarithmic complexity, not the base of the log.ŒˆEÊÍ4 6€€Ȁ€€†"€W‚ÿu Logarithms cut any function down to size: The growth rate of the logarithm of any polynomial function is . This follows because3HÌ7Í. ,€ €Ȁ†"€f‚ÿ`+Í—Î5 8€Y€Ȁ†"€P€€‚ÿThe power of binary search on a wide range of problems is a consequence of this observation. For example, note that doing a binary search on a sorted array of things requires only twice as many comparisons as a binary search on n things. Logarithms efficiently cut any function down to size.,7ÍÃÎ' € €€€‚ÿ)—ÎìÎ& €€€‚ÿ¿ŸÃη Ё€€‚çM8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çU8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€恩Û/€†"€‰€悩Û/€†"€‰€惩Û/€†"€‰€€çM8ۀ‰€€çfxÊ«€‰€€çU8ۀ‰€€‚‚ÿ ìη߇ Next: Modeling the Problem Up: Introduction to Algorithms Previous: Growth Rates AlgorithmsMon Jun 2 23:33:50 EDT 1997)ìÎà& €€€‚ÿµf·•O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1àÆ1Uÿÿÿÿÿÿÿÿ9ÿÿÿÿÆê$•ê" €€€ ÿEÆ/1 ÿÿÿÿÿÿÿÿ:/jîLModeling the Problem;êj# €0€€‚ÿModeling the Problem pk/Ú ؁瀀çN8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çL8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€愩Û/€†"€‰€慩Û/€†"€‰€憩Û/€†"€‰€€çN8ۀ‰€€çfxÊ«€‰€€çL8ۀ‰€‚€‚€‚ÿ Next: About the War Stories Up: Introduction to Algorithms Previous: LogarithmsModeling the ProblemModeling is the art of formulating your application in terms of precisely described, well-understood problems. Proper modeling is the key to applying algorithmic design techniques to any real-world problem. Indeed, proper modeling can eliminate the need to design or even implement algorithms by relating your application to what has been done before. Proper modeling is the key to effectively using the problem catalog in Part II of this book.  Cj% €=€€‚ÿReal-world applications involve real-world objects. You might be working on a system to route traffic in a network, to find the best way to schedule classrooms in a university, or to search for patterns in a corporate database. Most algorithms, however, are designed to work on rigorously defined abstract structures such as permutations, graphs, and sets. After all, if you can't define what you want to do, you can't hope to compute it. You must first describe your problem abstractly, in terms of fundamental structures and properties.Ì¡Úé + $€C€€€€‚ÿWhatever your application is, odds are very good that others before you have stumbled upon the same algorithmic problem, perhaps in substantially different contexts. Therefore, to find out what is known about your particular ``widget optimization problem,'' you can't hope to look in a book under widget. You have to formulate widget optimization in terms of computing properties of abstract structures such as:DÕ- o ¬€·€Ȁ€€€†"€g†"€h‚€€€†"€i†"€j†"€i†"€k‚ÿu Permutations, which are arrangements, or orderings, of items. For example, and are two distinct permutations of the same set of four integers. Permutations are likely the object in question whenever your problem seeks an ``arrangement,'' ``tour,'' ``ordering,'', or ``sequence.''       u Subsets, which represent selections from a set of items. For example, and are two distinct subsets of the first four integers. Order does not matter in subsets the way it does with permutations, so the subsets and would be considered identical. Subsets are likely the object in question whenever your problem seeks a ``cluster,'' ``collection,'' ``committee,'' ``group,'' ``packaging,'' or ``selection.''        ~Ié « 5 :€”€Ȁ†"€l€€‚ÿ  Figure: Modeling real-world structures with trees and graphs  ü™- ³@c ”€7€Ȁ€€€çAÕT+€†"€‰€‚€€€çAÕT+€†"€‰€‚ÿu Trees, which represent hierarchical relationships between items. Figure (a) illustrates a portion of the family tree of the Skiena clan.    Trees are likely the object in question whenever your problem seeks a ``hierarchy,'' ``dominance relationship,'' ``ancestor/decendant relationship,'' or ``taxonomy.''        u Graphs, which represent relationships between arbitrary pairs of objects. Figure (b) models a network of roads as a graph, where the vertices are cities and the « ³@êedges are roads connecting pairs of cities. Graphs are likely the object in question whenever you seek a ``network,'' ``circuit,'' ``web,'' or ``relationship.''     “^« FC5 8€œ€Ȁ€€‚€€€‚ÿu Points, which represent locations in some geometric space. For example, the locations of McDonald's restaurants can be described by points on a map/plane. Points are likely the object in question whenever your problems work on ``sites,'' ``positions,'' ``data records,'' or ``locations.''       u Polygons, which represent regions in some geometric space. For example, the borders of a country can be described by a polygon on a map/plane. Polygons and polyhedra are likely the object in question whenever you are working on ``shapes,'' ``regions,'' ``configurations,'' or ``boundaries.''       A³@‡D+ $€-€Ȁ€€‚ÿu Strings, which represent sequences of characters or patterns. For example, the names of students in a class can be represented by strings. Strings are likely the object in question whenever you are dealing with ``text,'' ``characters,'' ``patterns,'' or ``labels.''      LFCÓF1 0€7€€€€€€‚ÿThese fundamental structures all have associated problems and properties, which are presented in the catalog of Part II. Familiarity with all of these problems is important, because they provide the language we use to model applications. To become fluent in this vocabulary, browse through the catalog and study the input and output pictures for each problem. Understanding all or most of these problems, even at a cartoon/definition level, will enable you to know where to look later when the problem arises in your application.  V,‡D)J* "€Y€€‚€‚ÿExamples of successful application modeling will be presented in the war stories spaced throughout this book. However, some words of caution are in order. The act of modeling reduces your application to one of a small number of existing problems and structures. Such a process is inherently constraining, and certain details might not fit easily into the given model. Also, certain problems can be modeled in several different ways, some much better than others.Modeling is only the first step in designing an algorithm for a problem. Be alert for how the details of your applications differ from a candidate model. But don't be too quick to say that your problem is unique and special. Temporarily ignoring details that don't fit can free the mind to ask whether they were fundamental in the first place.)ÓFRJ& €€€‚ÿŸœ)JL Ё‹€€‚çN8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çL8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€懩Û/€†"€‰€戩Û/€†"€‰€扩Û/€†"€‰€€çN8ۀ‰€€çfxÊ«€‰€€çL8ۀ‰€€‚‚ÿ Next: About the War Stories Up: Introduction to Algorithms Previous: Logarithms AlgorithmsMon Jun 2 23:33:50 EDT 1997)RJ9L& €€€‚ÿµfLîLO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b19LM1Uÿÿÿÿÿÿÿÿ;ÿÿÿÿMCM$îLCM" €€€ ÿFM‰M1Ç ÿÿÿÿÿÿÿÿ<‰MÅM)‹About the War Stories<CMÅM# €2€€‚ÿAbout the War Stories ©€‰Mz€ ؁Y€€çO8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çM8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€浩Û/€†"€‰€欩Û/€†"€‰€æ­©Û/€†"€‰€€çO8ۀ‰€€çfxÊ«€‰€€çM8ۀ‰€‚€‚€‚ÿ Next: War Story: Psychic Modeling Up: Introduction to Algorithms Previous: Modeling the ProblemAbout the War StoriesThe best way to become convinced that careful algorithm design can have a significant impact on performance is to look at ÅMz€CMcase studies. By carefully studying other people's experiences, we learn how they might apply to our work.  2ÅMƒ[ „€e€€€€€€çOž;C€‰€çPž;C€‰€€€çў;C€‰€‚ÿJon Bentley's Programming Pearls columns are probably the most interesting collection of algorithmic ``war stories'' currently available. Originally published in the Communications of the ACM, they have been collected in two books [Ben86, Ben90]. Fredrick Brooks's The Mythical Man Month [Bro74] is another wonderful collection of war stories, focused more on software engineering than algorithm design, but they remain a source of considerable wisdom. Every programmer should read all three of these books, for pleasure as well as insight.¢uz€©…- (€ë€€€‚€‚ÿScattered throughout this text are several of our own war stories, presenting our successful (and occasionally unsuccessful) algorithm design efforts on real applications. We hope that the reader will be able to internalize these experiences so that they will serve as models for their own attacks on problems. Every one of the war stories is true. Of course, the stories improve somewhat in the retelling, and the dialogue has been punched up to make it more interesting to read. However, I have tried to honestly trace the process of going from a raw problem to a solution, so you can watch how the process unfolded.   «uƒTˆ6 :€ë€€€€€€‚€‚ÿThe Oxford English Dictionary defines an algorist as ``one skillful in reckonings or figuring.'' In these stories, I have tried to capture some of the attitude, or mindset, of the algorist in action as they attack a problem.The various war stories usually involve at least one and often several problems from the problem catalog in Part II. The appropriate section of the catalog is referenced wherever it occurs. This emphasizes the benefits of modeling your application in terms of standard algorithm problems. By using the catalog, you will be able to pull out what is known about any problem whenever it is needed.)©…}ˆ& €€€‚ÿÎÍTˆKŠ Ё«€€‚çO8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çM8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€殩Û/€†"€‰€毩Û/€†"€‰€æ°©Û/€†"€‰€€çO8ۀ‰€€çfxÊ«€‰€€çM8ۀ‰€€‚‚ÿ Next: War Story: Psychic Modeling Up: Introduction to Algorithms Previous: Modeling the Problem AlgorithmsMon Jun 2 23:33:50 EDT 1997)}ˆtŠ& €€€‚ÿµfKŠ)‹O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1tŠZ‹1Uÿÿÿÿÿÿÿÿ=ÿÿÿÿZ‹~‹$)‹~‹" €€€ ÿLZ‹ʋ1¿.ÿÿÿÿÿÿÿÿ>ʋ ŒžGWar Story: Psychic ModelingB~‹ Œ# €>€€‚ÿWar Story: Psychic Modeling ž•ʋªŽ  à;€€çP8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çN8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€汩Û/€†"€‰€沩Û/€†"€‰€泩Û/€†"€‰€€çP8ۀ‰€€çfxÊ«€‰€€çN8ۀ‰€‚€‚€‚‚‚ÿ Next: Exercises Up: Introduction to Algorithms Previous: About the War StoriesWar Story: Psychic Modeling The call came for me out of the blue as I sat in my office.``Professor Skiena, I hope you can help me. I'm the President of Lotto Systems Group Inc., and we need an algorithm for a problem arising in our latest product.''¢f ŒXÁ< F€Ï€€‚ç=ûÏš€†"€m‰€‚ÿ``Sure,'' I replied. After all, the dean of my engineering school is always eager for our faculty to interact with industry.    ``At Lotto Systems Group, we market a program designed to improve our customers' psychic ability to predict winning lottery numbers. In a standarªŽXÁ~‹d lottery, each ticket consists of 6 numbers selected from, say, 1 to 44. Thus any given ticket has only a very small chance of winning. However, after proper training, our clients can visualize a set of, say, 15 numbers out of the 44 and be certain that at least 4 of them will be on the winning ticket. Are you with me so far?''\5ªŽŽÃ' €k€€‚‚ÿ``Probably not,'' I replied. But then I recalled how my dean wants us to interact with industry.``Our problem is this. After the psychic has narrowed the choices down to 15 numbers and is certain that at least 4 of them will be on the winning ticket, we must find the most efficient way to exploit this information. Suppose that a cash prize is awarded whenever you pick at least three of the correct numbers on your ticket. We need an algorithm to construct the smallest set of tickets that we must buy in order to guarantee that we win at least one prize.''m)XÁ!ÇD V€U€€‚‚€€繿 耆"€‰€‚ÿ``Assuming the psychic is correct?''``Yes, assuming the psychic is correct. We need a program that prints out a list of all the tickets that the psychic should buy in order to minimize their investment. Can you help us?''I thought about the problem for a minute. Maybe they did have psychic ability, for they had clearly come to the right place. Identifying the best subset of tickets to buy was very much a combinatorial algorithm problem. It was going to be some type of covering problem, where each ticket we buy was going to ``cover'' some of the possible 4-element subsets of the psychic's set. Finding the absolute smallest set of tickets to cover everything was a special instance of the NP-complete problem set cover (discussed in Section ) and presumably computationally intractable.  /ŽÃ#ÊÓ tq€€†"€n€€†"€o†"€p†"€q†"€r†"€s‚€€€€†"€t€€†"€u€€€€€€€€€€çÿ€ÙY€†"€‰€€€€€€€€€‚ÿ  Figure: Covering all pairs of with tickets , , ,  It was indeed a special instance of set cover, completely specified by only four numbers: the size n of the candidate set S (typically ), the number of k slots for numbers on each ticket (typically ), the number of psychically promised correct numbers j from S (say j=4), and finally, the number of numbers l matching the winning ticket necessary to win a prize (say l=3). Figure illustrates a covering of a smaller instance, where n=5, k=3, and j=l=2.k<!ÇŽÌ/ ,€y€€€€‚‚‚ÿ``Although it will be hard to find the exact minimum set of tickets to buy, with heuristics I should be able to get you pretty close to the cheapest covering ticket set,'' I told him. ``Will that be good enough?''``So long as it generates better ticket sets than my competitor's program, that will be fine. His system doesn't always provide enough coverage to guarantee a win. I really appreciate your help on this, Professor Skiena.''``One last thing. If your program can train people to pick lottery winners, why don't you use it to win the lottery yourself?''„]#ÊÎ' €»€€‚‚ÿ``I look forward to talking to you again real soon, Professor Skiena. Thanks for the help.''I hung up the phone and got back to thinking. It seemed like a fun problem, and the perfect project to give to a bright undergraduate. After it was modeled in terms of sets and subsets, the basic components of a solution seemed fairly straightforward:VíŽÌti  €ß€Ȁ€€€€€€ç `4"€†"€‰€‚€€†"€v€€€€‚ÿu We needed the ability to generate all subsets of k numbers from the candidate set S. Algorithms for generating and ranking/unranking subsets of sets are presented in Section .    u We needed the right formulation of what it meant to have a covering set of purchased tickets. The obvious criteria would be to pick a small set of tickets such that for each of the l-subÎt~‹sets of S that might pay off with the prize, we have purchased at least one ticket that contains it.À}Î4C T€ý€Ȁ€çÕ•€†"€‰€€€‚ÿu For each new ticket we decided to buy, we would need to keep track of which prize combinations we have thus far covered. Presumably, we would want to buy tickets to cover as many thus-far-uncovered prize combinations as possible. The currently covered combinations are a subset of all possible combinations, so we would need a data structure to represent the subset. Such data structures are discussed in Section . The best candidate seemed to be a bit vector, which would permit a constant time query of ``is this combination already covered?'' in concert with a way to rank the subset, i.e. hash it to a unique integer.   F tz= H€€Ȁ€ç Æ•ù€†"€‰€‚ÿu We would need a search mechanism to decide which ticket to buy next. For small enough set sizes, we could do an exhaustive search over all possible subsets of tickets and pick the smallest one. For larger sizes, a randomized search process like simulated annealing (see Section ) would select tickets-to-buy to cover as many uncovered combinations as possible. By repeating this randomized procedure several times and picking the best solution, we would be likely to come up with a very good set of tickets.   †b4$ €Ä€€‚ÿExcluding the search mechanism, the pseudocode for the bookkeeping looked something like this:tÙzt › œ€€‚€€‚†"€v€€‚€€‚€€€€†"€w‚€€†"€x€€†"€y‚‚€†"€z‚ÿ LottoTicketSet(n,k,l) Initialize the -element bit-vector V to all false While there exists a false entry of V Select a k-subset T of as the next ticket to buy For each of the l-subsets of T, Report the set of tickets bought The bright undergraduate, Fayyaz Younas, rose to the challenge. Based on this framework, he implemented a brute-force search algorithm and found optimal solutions for problems with in a reasonable time. To solve larger problems, he implemented a random search procedure, tweaking it for a while before settling on the best variant. Finally, the day arrived when we could call Lotto Systems Group and announce that we had solved the problem.i| Ÿ å€€€€€€€€€€‚€€†"€{†"€|†"€}†"€~†"€‚†"€€€€†"€o†"€p†"€q‚ÿ``See, our program found that optimal solution for n=15, k=6, j=6, l=3 meant buying 28 tickets.''``Twenty-eight tickets!'' complained the president. ``You must have a bug. Look, these five tickets will suffice to cover everything twice over: , , , , .''  Figure: Guaranteeing a winning ticket from using only tickets and  XÜt Ô| ƀÇ€€€€çØ©ˆ€†"€‰€†"€†"€p†"€‚†"€ƒ†"€„€€†"€„‚ÿWe fiddled with this example for a while before admitting that he was right. We hadn't modeled the problem correctly! In fact, we didn't need to explicitly cover all possible winning combinations. Figure illustrates the principle by giving a better solution to the previous small example. As a different example, suppose we needed three numbers on a five element ticket to win, and the psychic had promised us that four numbers would be from . Suppose that we had bought tickets that covered the subsets , , and . There would be no need to buy a ticket that explicitly covered , because if 1 and 2 and two other psychic numbers were on a winning ticket, then at least one of the other two psychic numbers would have to be either 3, 4, or 5. If 1 and 2 were not on the winning ticket, a ticket covering would do us absolutely no good. Thus we were trying to cover too many combinations, and the penny-pinching psychics were unwilling to pay for such extravagance.(÷| B1 0€ï€€çמK€‰ÔB~‹€‚ÿFortunately, this story has a happy ending, as reported in [YS96]. The general outline of our search-based solution described above still holds for the real problem. All that is needed to fix it is to change how many uncovered subsets we credit for covering with a given set of tickets. After this modification, we obtained the kind of results they were hoping for. Lotto Systems Group gratefully accepted our program to incorporate into their products, and we hope they hit the jackpot with it.Ò€ÔÚD. *€I€€€€€‚ÿThe moral of this story is to make sure that you correctly model the problem before trying to solve it. In our case, we came up with a reasonable model for the problem but didn't work hard enough to validate it before we started to program. Our misinterpretation would have become obvious had we worked out a small but non-trivial example by hand and bounced it off our sponsor before beginning work. Our success in recovering from the error is a tribute to the basic correctness of our initial formulation and our use of well-defined routines and abstractions for such tasks as (1) ranking/unranking k-subsets, (2) the set data structure, and (3) combinatorial search.)BE& €€€‚ÿœŒÚDÀF Ё‰€€‚çP8ۀ†"€ ‰€çfxÊ«€†"€ ‰€çN8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€掩Û/€†"€‰€æà©Û/€†"€‰€æשÛ/€†"€‰€€çP8ۀ‰€€çfxÊ«€‰€€çN8ۀ‰€€‚‚ÿ Next: Exercises Up: Introduction to Algorithms Previous: About the War Stories AlgorithmsMon Jun 2 23:33:50 EDT 1997)EéF& €€€‚ÿµfÀFžGO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1éFÏG1Uÿÿÿÿÿÿÿÿ?ÿÿÿÿÏGóG$žGóG" €€€ ÿ: ÏG-H1«ÿÿÿÿÿÿÿÿ@-H]HŒExercises0 óG]H# €€€‚ÿExercises œž-HJ ؁€€ç@n_ɀ†"€ ‰€çfxÊ«€†"€ ‰€çO8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æØ©Û/€†"€‰€æÙ©Û/€†"€‰€æÚ©Û/€†"€‰€€ç@n_ɀ‰€€çfxÊ«€‰€€çO8ۀ‰€‚€‚€‚ÿ Next: Implementation Challenges Up: Introduction to Algorithms Previous: War Story: Psychic ModelingExercises i]HƒK[ „€!€Ȁ€€€€†"€€€†"€…€€€€€€‚ÿ1. Let P be a problem. The worst-case time complexity of P is . The worst-case time complexity of P is also . Let A be an algorithm that solves P. Which subset of the following statements are consistent with this information about the complexity of P?cÛJæLˆ ހ¿€‘€€€€€†"€‚€€€†"€†‚€€€€€€€‚€€€†"€‡‚€€€†"€ˆ‚ÿu A has worst-case time complexity .u A has worst-case time complexity .u A has worst-case time complexity O(n).u A has worst-case time complexity .u A has worst-case time complexity .9ރKN[ „€œ€Ȁ€€€€€€€€€€€€€€€€€€‚ÿ2. Suppose that an algorithm A runs in worst-case time f(n) and that algorithm B runs in worst-case time g(n). For each of the following questions, answer either yes, no, or can't tell and explain why.ŽEæL­NI b€Ž€Ȁ€€€€€€†"€K†"€‰‚ÿ(a) Is A faster than B for all n greater than some if ?ŽEN;OI b€Ž€Ȁ€€€€€€†"€K†"€Š‚ÿ(b) Is A faster than B for all n greater than some if ?ŽE­NÉOI b€Ž€Ȁ€€€€€€†"€K†"€‹‚ÿ(c) Is A faster than B for all n greater than some if ?ŽE;Oc€I b€Ž€Ȁ€€€€€€†"ÉOc€óG€K†"€‰‚ÿ(d) Is B faster than A for all n greater than some if ?ŽEÉOñ€I b€Ž€Ȁ€€€€€€†"€K†"€Š‚ÿ(e) Is B faster than A for all n greater than some if ?͂c€ŸK d€ €Ȁ€€€€€€†"€K†"€‹‚‚ÿ(f) Is B faster than A for all n greater than some if ?3. For each of these questions, briefly explain your answer.­sñ€k‚: D€è€Ȁ†"€€€€€‚ÿ(a) If I prove that an algorithm takes worst-case time, is it possible that it takes O(n) on some inputs?¬rŸƒ: D€æ€Ȁ†"€€€€€‚ÿ(b) If I prove that an algorithm takes worst-case time, is it possible that it takes O(n) on all inputs?­sk‚ă: D€è€Ȁ†"€‡€€€€‚ÿ(c) If I prove that an algorithm takes worst-case time, is it possible that it takes O(n) on some inputs?¬rƒp„: D€æ€Ȁ†"€‡€€€€‚ÿ(d) If I prove that an algorithm takes worst-case time, is it possible that it takes O(n) on all inputs?þžÄƒn…` Ž€C€Ȁ†"€Œ†"€€€†"€Ž€€‚€€€€€€‚ÿ(e) Is the function , where for even n and for odd n?4. For each of the following, answer yes, no, or can't tell. Explain your reasoning!<p„ª…. ,€€Ȁ†"€‚ÿ(a) Is ?<n…æ…. ,€€Ȁ†"€‚ÿ(b) Is ?<ª…"†. ,€€Ȁ†"€‘‚ÿ(c) Is ?Ƨæ…èˆ ‚S€Ȁ†"€’‚€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚€€€€€€€€€€†"€“‚€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿ(d) Is ?5. (*) Give a proof or counterexample to the following claim: if f(n) = O(F(n)) and g(n) = O(G(n)), then f(n)/g(n) = O(F(n)/G(n)).6. (*) Does f(n) = O(g(n)) imply that ? Explain your reasoning!7. (*) Give a proof or counterexample to the following claim: for all functions f(n) and g(n), either f(n) = O(g(n)) or g(n) = O(f(n)).MÚ"†5‹s Ž€¹€Ȁ†"€”€€€€†"€•€€€€€€€€€€€€€€€€‚ÿ8. (*) When you first learned to multiply numbers, you were told that means add x a total of y times, so . What is the time complexity of multiplying two n-digit numbers in base b (people work in base 10, of course, while computers work in base 2) using the repeated addition method, as a function of n and b. Assume that single-digit by single-digit addition or multiplication takes O(1) time. (hint: how big can y be as a function of n and b?)  A舶Œ@ N€…€Ȁ†"€–€€€€€€‚ÿ9. (*) In grade school, you learned to multiply long numbers on a digit by digit basis, so that . Analyze the time complexity of multiplying two n-digit numbers with this method as a function of n (assume constant base size). Assume that single-digit by single-digit addition or multiplication takes O(1) time.,5‹âŒ' € €€€‚ÿ)¶Œ & €€€‚ÿÓÒâŒގ Ёµ€€‚ç@n_ɀ†"€ ‰€çfxÊ«€†"€ ‰€çO8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÛ©Û/€†"€‰€æÜ©Û/€†"€‰€æÝ©Û/€†"€‰€€ç@n_ɀ‰€€çfxÊ«€‰€€çO8ۀ‰€€‚‚ÿ Next: Implementation Challenges Up: Introduction to Algorithms Previous: War Story: Psychic Modeling AlgorithmsMon Jun 2 23:33:50 EDT 1997) & €€€‚ÿµfގŒO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 À1UÿÿÿÿÿÿÿÿAÿÿÿÿ À0ÀŒ ÀŒ$Œ0À" €€€ ÿG ÀwÀ1® ÿÿÿÿÿÿÿÿBwÀŽÀ1ÊFundamental Data Types=0ÀŽÀ# €4€€‚ÿFundamental Data Types ÖÑwÀŠÃ ؁³€€çR8ۀ†"€ ‰€çgxÊ«€†"€ ‰€çgxÊ«€†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÞ©Û/€†"€‰€æß©Û/€†"€‰€æ ªÛ/€†"€‰€€çR8ۀ‰€€çgxÊ«€‰€€çgxÊ«€‰€‚€‚€‚ÿ Next: Containers Up: Data Structures and Sorting Previous: Data Structures and SortingFundamental Data TypesAn abstract data type is a collection of well-defined operations that can be performed on a particular structure. The operations define what the data type does, but not how it works. Abstract data types are black boxes that we dare not open when we design algorithms that use them.  ×ŽÀŠÇE X€±€€€€‚çq÷ z€†"€‰€€‚ÿFor each of the most important abstract data types, several competing implementations, or data structures, are available. Often, alternative data structures realize different design tradeoffs that make certain operations (say, insertion) faster at the cost of other operations (say, search). In some applications, certain data structures yield simpler code than other implementations of a given abstract data type, or have some other specialized advantages.We assume that the reader has had some previous exposure to elementary data structures and some fluency in pointer manipulation. Therefore, we do not discuss these topics here. The reader who wants to review elementary data structures is referred to any of the books in Section . Instead, we focus on three fundamental abstract data types: containers, dictionaries, and priority queues. Detailed discussion of the tradeoffs between implementations of these data types is deferred to the relevant catalog entry.  )ŠÃÏÇ& €€€‚ÿ&ŠÇõÇ# €€€‚ÿŒ\ÏDZÈ` €ž€Ȁç%,ív€‰€‚€熺óՀ‰€‚€ççHú4€‰€‚€ç“ÁŠ€‰€‚ÿu Containers u Dictionaries u Binary Search Trees u Priority Queues&õÇ×È# €€€‚ÿ)±ÈÉ& €€€‚ÿS+×ÈSÉ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)É|É& €€€‚ÿµfSÉ1ÊO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1|ÉbÊ1UÿÿÿÿÿÿÿÿCÿÿÿÿbʆÊ$1ʆÊ" €€€ ÿ; bÊÁÊ1QÿÿÿÿÿÿÿÿDÁÊòÊ Containers1†ÊòÊ# €€€‚ÿContainers &ÁÊÎ ð;€€çS8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æªÛ/€†"€‰€æªÛ/€†"€‰€æªÛ/€†"€‰€€çS8ۀ‰€€çQ8ۀ‰€€çQ8ۀ‰€‚€ ‚€€€€€‚ÿ Next: Dictionaries Up: Fundamental Data Types Previous: Fundamental Data TypesContainersContainers are abstract data types that hold stuff. They don't do much more than hold it so that it can be retrieved later. Still, they are critical to the functioning of society. We will use the term container to denote a data structure that permits storage and retrieval of data items independently of content. The two fundamental operations of any container are:  4éòÊLÏK d€Ó€Ȁ€€€€€€€‚€€€€€‚ÿu Put(C,x): Insert a new data item x into the container C.u Get(C): Retrieve the next item from the container C. Different types of containers support different retrieval orders, based on insertion order or position.æÁÎ>% €ƒ€€‚ÿContainers are typically most useful when they will contain only a limited number of items and when the retrieval order is predefined or irreLÏ>†Êlevant. The most popular type of containers are:gþLÏ¥i  €ý€Ȁ€€€€€€€€€€€‚€€€€€€€€€€€‚ÿu Stacks: Supports retrieval in last in, first out order (LIFO). Stacks are simple to implement, and very efficient. Indeed, stacks are probably the right container to use when the retrieval order doesn't matter at all, as when processing batch jobs. The put and get operations for stacks are usually called push and pop.    u Queues: Supports retrieval in first in, first out order (FIFO). FIFO may seem the fairest way to control waiting times. However, for many applications, data items have infinite patience. Queues are trickier to implement than stacks and are appropriate only for applications (like certain simulations) where the order is important. The put and get operations for queues are usually called enqueue and dequeue.    Óœ>x7 <€9€Ȁ€€€€€€‚ÿu Tables: Supports retrieval by position, so that put and get each accept an index as an argument. Tables are naturally implemented using arrays.  â®¥Z4 6€]€€€€€€€‚ÿEach of these containers can be implemented using either arrays or linked lists. With the exception of tables, the choice of lists versus tables probably doesn't matter very much. The key issue is whether an upper bound on the size of the container is known in advance, thus permitting a statically allocated array. Using arrays, put and get can be implemented in constant time per operation for each of the containers.  )xƒ& €€€‚ÿœŒZ@ Ё‰€€‚çS8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æªÛ/€†"€‰€æªÛ/€†"€‰€æªÛ/€†"€‰€€çS8ۀ‰€€çQ8ۀ‰€€çQ8ۀ‰€€‚‚ÿ Next: Dictionaries Up: Fundamental Data Types Previous: Fundamental Data Types AlgorithmsMon Jun 2 23:33:50 EDT 1997)ƒi& €€€‚ÿµf@ O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1iO 1UÿÿÿÿÿÿÿÿEÿÿÿÿO s $ s " €€€ ÿ= O ° 1!ÿÿÿÿÿÿÿÿF° ã µGDictionaries3s ã # € €€‚ÿDictionaries TM° 7  ܁«€€çT8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çR8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æªÛ/€†"€‰€æ ªÛ/€†"€‰€æ ªÛ/€†"€‰€€çT8ۀ‰€€çQ8ۀ‰€€çR8ۀ‰€‚€ ‚€‚‚ÿ Next: Binary Search Trees Up: Fundamental Data Types Previous: ContainersDictionaries Dictionaries are a form of container that permits access to data items by content. You put a word into a dictionary because you know you can look it up when you need it.  X4㠏 $ €h€€‚ÿThe primary operations dictionaries support are:ºG7 Is Ž€€Ȁ€€€€€€€€€‚€€€€€€€‚€€€€€€€€€‚ÿu Search(D,k): Given a search key k, return a pointer to the element in dictionary D whose key value is k, if one exists.u Insert(D,x): Given a data item x, add it to the set of items in the dictionary D.u Delete(D,x): Given a pointer to a given data item x in the dictionary D, remove it from D.$ö y@. *€ï€€†"€W‚ÿPerhaps the simplest possible dictionary implementation maintains an unsorted linked list as a data structure. Insertion and deletion are supported in constant time, although a query requires potentially traversing the entire linked list. Basing an implementation on a stored array speeds up the query operation to by binary search. Making room for a new item or filling a hole left by a dIy@s eletion may require moving arbitrarily many items, so insertion and deletion become linear-time operations./óIšB< F€é€€çЅـ†"€‰€‚‚ÿMany other dictionary implementations are available. Binary search trees are discussed in some detail in the next section. Hash tables are another attractive option in practice. A complete discussion of different dictionary data structures is presented catalog Section . We encourage the reader to browse through the data structures section of the catalog in order to learn what your options are.Certain dictionary data structures also efficiently support the following useful operations:"¹y@ÊDi  €s€Ȁ€€€€€€€‚€€€€€€€€€€€€€€€‚ÿu Max(D) or Min(D): Retrieve the item with the largest (or smallest) key from D. This enables the dictionary to serve as a priority, as discussed below.u Predecessor(D,k) or Successor(D,k): Retrieve the item from D whose key is immediately before (or after) k in sorted order. By starting from the first item Min(D) and repeatedly calling Successor until we obtain Max(D), we traverse all elements in sorted order.,šBöD' € €€€‚ÿ)ÊDE& €€€‚ÿž·öD×F Ё€€‚çT8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çR8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ6ªÛ/€†"€‰€æ-ªÛ/€†"€‰€æ.ªÛ/€†"€‰€€çT8ۀ‰€€çQ8ۀ‰€€çR8ۀ‰€€‚‚ÿ Next: Binary Search Trees Up: Fundamental Data Types Previous: Containers AlgorithmsMon Jun 2 23:33:50 EDT 1997)EG& €€€‚ÿµf×FµGO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1GæG1UÿÿÿÿÿÿÿÿGÿÿÿÿæG H$µG H" €€€ ÿDæGNH1¶ÿÿÿÿÿÿÿÿHNHˆH¹ŒBinary Search Trees: HˆH# €.€€‚ÿBinary Search Trees W%NHßK2 2‚]€€ç€8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çS8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ/ªÛ/€†"€‰€æ0ªÛ/€†"€‰€æ1ªÛ/€†"€‰€€ç€8ۀ‰€€çQ8ۀ‰€€çS8ۀ‰€‚€ ‚€€€çàHú4€†"€‰€€€€€€€‚ÿ Next: Priority Queues Up: Fundamental Data Types Previous: DictionariesBinary Search TreesFast support of all dictionary operations is realized by binary search trees. A binary tree is a rooted tree where each node contains at most two children. Each child can be identified as either a left or right child. As shown in Figure , a binary tree can be implemented where each node has left and right pointer fields, an (optional) parent pointer, and a data field.  ž%ˆH}Oy À€M€€†"€—€€‚€€€€€€€€€€€€€€€€€€€€€€‚ÿ  Figure: Relationships in a binary search tree  A binary search tree labels each node in a binary tree with a single key such that for any node labeled x, all nodes in the left subtree of x have keys < x while all nodes in the right subtree of x have keys > x. The search tree labeling enables us to find where any key is. Start at the root. If it does not contain the key we are searching for, proceed either left or right depending upon whether what we want occurs before or after the root key. This algorithm works because both the left and right subtrees of a binary search tree are binary search trees; the recursive structure yields a recursive algorithm. Accordingly, the dictionary Query operation can be performed in O(h) time, where h is the height of the tree.&gßK¯‚¿ LÑ€€‚€€‚†"€˜‚€€‚€€€€€€‚€€€€€€‚€€€€€}O¯‚ H€‚€€€€€€€€€€€€€€€€€‚ÿ BinaryTreeQuery(x, k) if then return x if (k < key[x]) then return BinaryTreeQuery(left[x],k) else return BinaryTreeQuery(right[x],k) To insert a new item x with key k into a binary search tree T, it is important to place it where it can be later be found. There is only one such location in any binary search tree, namely by replacing the nil pointer found in T after an unsuccessful query for k. Replacing this nil pointer with a pointer to x is a simple, constant-time operation after the search has been performed in O(h) time.  6ë}Oå†K d€Û€€‚€€€€€€†"€™†"€W‚ÿDeletion is somewhat more tricky than insertion, because the node selected to die may not be a leaf. Leaf nodes may be deleted without mercy, by clearing the pointer to the given node. However, internal nodes have children that must remain accessible after the deletion. By restructuring or relabeling the tree, however, the item to delete can always be made into a leaf and then removed. Details appear in any data structures text.  When implemented using binary search trees, all three dictionary operations take O(h) time, where h is the height of the tree. The smallest height we could hope for occurs when the tree is perfectly balanced, where . In fact, if we insert the keys in random order, with high probability the tree will have height. However, if we get unlucky with our order of insertion or deletion, we can end up with a linear-height tree in the worst case. The worst case is a serious potential problem. Indeed, it occurs whenever the keys are inserted in sorted order.À¯‚ü‰W |€‡€€€€†"€W†"€WçЅـ†"€‰€‚€‚ÿTo avoid such worst-case performance, more sophisticated balanced binary search tree data structures have been developed that guarantee the height of the tree always to be . Therefore, all dictionary operations (insert, delete, query) take time each. Implementations of such balanced tree data structures as red-black trees are discussed in Section .  From an algorithm design viewpoint, it is most important to know that these trees exist and that they can be used as black boxes to provide an efficient dictionary implementation. When figuring the costs of dictionary operations for algorithm analysis, assume the worst-case complexities of balanced binary trees to be a fair measure.)å†%Š& €€€‚ÿ¶µü‰ۋ Ё{€€‚ç€8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çS8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ2ªÛ/€†"€‰€æ3ªÛ/€†"€‰€æ4ªÛ/€†"€‰€€ç€8ۀ‰€€çQ8ۀ‰€€çS8ۀ‰€€‚‚ÿ Next: Priority Queues Up: Fundamental Data Types Previous: Dictionaries AlgorithmsMon Jun 2 23:33:50 EDT 1997)%ŠŒ& €€€‚ÿµfۋ¹ŒO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ŒêŒ1UÿÿÿÿÿÿÿÿIÿÿÿÿꌍ$¹Œ" €€€ ÿ@êŒN1¶ÿÿÿÿÿÿÿÿJN„ºÍPriority Queues6„# €&€€‚ÿPriority Queues N—À ؁€€çw8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çT8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ5ªÛ/€†"€‰€æaªÛ/€†"€‰€æXªÛ/€†"€‰€€çw8ۀ‰€€çQ8ۀ‰€€çT8ۀ‰€‚€ ‚€‚ÿ Next: Specialized Data Structures Up: Fundamental Data Types Previous: Binary Search TreesPriority QueuesMany algorithms process items according to a particular order. For example, suppose you have to schedule a list of jobs given the deadline by which each job must be performed or else its importance „—Àrelative to the other jobs. Scheduling jobs requires sorting them by time or importance, and then performing them in this sorted order.  Ð©„gÂ' €S€€‚‚ÿPriority queues provide extra flexibility over sorting, which is required because jobs often enter the system at arbitrary intervals. It is much more cost-effective to insert a new job into a priority queue than to re-sort everything. Also, the need to perform certain jobs may vanish before they are executed, meaning that they must be removed from the queue.The basic priority queue supports three primary operations:ßr—ÀFÄm š€å€Ȁ€€€€€€€€€‚€€€€€€€‚€€€€€€€‚ÿu Insert(Q,x): Given an item x with key k, insert it into the priority queue Q.u Find-Minimum(Q) or Find-Maximum(Q): Return a pointer to the item whose key value is smaller (larger) than any other key in the priority queue Q.u Delete-Minimum(Q) or Delete-Maximum(Q) - Remove the item from the priority queue Q whose key is minimum (maximum).ËygÂÇR r€÷€€†"€š€€‚€€çŒÁŠ€†"€‰€‚ÿ  Figure: The maximum and minimum element in a binary search tree  All three of these priority queue operations can be implemented in time by representing the heap with a binary search tree. Implementing the find-minimum operation requires knowing where the minimum element in the tree is. By definition, the smallest key must reside in the left subtree of the root, since all keys in the left subtree have values less than that of the root. Therefore, as shown in Figure , the minimum element must be the leftmost decendent of the root. Similarly, the maximum element must be the rightmost decendent of the root.  »+FÄÌɐ î€[€€‚€€€€‚‚€€€€€€€€€€€€‚€€€€‚€†"€W€€†"€W‚ÿ Find-Maximum(x) Find-Minimum(x) while while do x = right[x] do x = left[x] return x return x Repeatedly traversing left (or right) pointers until we hit a leaf takes time proportional to the height of the tree, or if the tree is balanced. The insert operation can be implemented exactly as binary tree insertion. Delete-Min can be implemented by finding the minimum element and then using standard binary tree deletion. It follows that each of the operations can be performed in time.ÌÇêÊR r€€€çÖ«4€€†"€‰€ç|þ4.€†"€‰€€‚ÿPriority queues are very useful data structures. Indeed, they are the hero of the war story described in Section . A complete set of priority queue implementations is presented in Section .)ÌÉË& €€€‚ÿÉÈêÊÜÌ Ё¡€€‚çw8ۀ†"€ ‰€çQ8ۀ†"€ ‰€çT8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æYªÛ/€†"€‰€æZªÛ/€†"€‰€æ[ªÛ/€†"€‰€€çw8ۀ‰€€çQ8ۀ‰€€çT8ۀ‰€€‚‚ÿ Next: Specialized Data Structures Up: Fundamental Data Types Previous: Binary Search Trees AlgorithmsMon Jun 2 23:33:50 EDT 1997)ËÍ& €€€‚ÿµfÜ̺ÍO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÍëÍ1UÿÿÿÿÿÿÿÿKÿÿÿÿëÍÎ$ºÍÎ" €€€ ÿLëÍ[Î1ÎÿÿÿÿÿÿÿÿL[ΝÎÊ Specialized Data StructuresBΝÎ# €>€€‚ÿSpecialized Data Structures  [Ξ ؁%€€çx8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç€8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ\ªÛ/€†"€‰€æ]ªÛ/€†"€‰€æ^ªÛ/€†"€‰€€çx8ۀ‰€€çgxÊ«€‰€€ç€8ۀ‰€‚€‚€‚ÿ Next: Sorting Up: Data Structures aÎžÎnd Sorting Previous: Priority QueuesSpecialized Data StructuresThe basic data structures thus far described all represent an unstructured set of items so as to facilitate retrieval operations. These data structures are well known to most programmers. Not as well known are high-powered data structures for representing more structured or specialized kinds of objects, such as points in space, strings, and graphs.õНέ% €¡€€‚ÿThe design principles of these data structures are the same as for basic objects. There exists a set of basic operations we need to perform repeatedly. We seek a data structure that supports these operations very efficiently. These efficient, specialized data structures are as important for efficient graph and geometric algorithms as lists and arrays are for basic algorithms, so one should be aware of their existence. Details appear throughout the catalog.¡&žN{ ĀU€Ȁ€€€çýùր†"€‰€‚€€€†"€›†"€œ€€ç6š ô€†"€‰€‚ÿu String data structures - Character strings are typically represented by arrays of characters, with perhaps a special character to mark the end of the string. Suffix trees/arrays are special data structures that preprocess strings to make pattern matching operations faster. See Section for details.   u Geometric data structures - Geometric data typically consists of collections of data points and regions. Regions in the plane are usually described by polygons, where the boundary of the polygon is given by a chain of line segments. Polygons can be represented using an array of points , such that is a segment of the boundary. Spatial data structures such as kd-trees organize points and regions by geometric location to support fast search. See Section for details.     “­á t ¶€E€Ȁ€€ç@’P€†"€‰€çt‹6€†"€‰€‚€€€çÕ•€†"€‰€‚ÿu Graph data structures - Graphs are typically represented by either adjacency matrices or adjacency lists. The choice of representation can have a substantial impact on the design of the resulting graph algorithms, as discussed in Chapter . Implementation aspects of graph data structures are presented in Section .  u Set data structures - Subsets of items are typically represented using a dictionary, to support fast membership queries. A variety of data structures for manipulating sets is presented in Section .  ,N ' € €€€‚ÿ)á 6 & €€€‚ÿ¶µ ì  Ё{€€‚çx8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç€8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ_ªÛ/€†"€‰€æ`ªÛ/€†"€‰€æűÛ/€†"€‰€€çx8ۀ‰€€çgxÊ«€‰€€ç€8ۀ‰€€‚‚ÿ Next: Sorting Up: Data Structures and Sorting Previous: Priority Queues AlgorithmsMon Jun 2 23:33:50 EDT 1997)6  & €€€‚ÿµfì Ê O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 û 1UÿÿÿÿÿÿÿÿMÿÿÿÿû  $Ê  " €€€ ÿ8û W 1C ÿÿÿÿÿÿÿÿNW … oFSorting.  … # €€€‚ÿSorting ßÚW p@ ؁Å€€çy8ۀ†"€ ‰€çgxÊ«€†"€ ‰€çw8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€挱Û/€†"€‰€朱Û/€†"€‰€柱Û/€†"€‰€€çy8ۀ‰€€çgxÊ«€‰€€çw8ۀ‰€‚€‚€‚ÿ Next: Applications of Sorting Up: Data Structures and Sorting Previous: Specialized Data StructuresSortingBy the time they graduate, computer science students are likely to have studied the basic sorting algorithms in their introductory programming class, then in their data structures class, and final… p@ ly in their algorithms class. Why is sorting worth so much attention? There are several reasons:  "ã… ’B? L€Ç€Ȁ€€‚€€琑K€‰€‚ÿu Sorting is the basic building block around which many other algorithms are built. By understanding sorting, we obtain an amazing amount of power to solve other problems.u Historically, computers have spent more time sorting than doing anything else. A quarter of all mainframe cycles are spent sorting data [Knu73b]. Although it is unclear whether this remains true on smaller computers, sorting remains the most ubiquitous combinatorial algorithm problem in practice.Wp@éD; D€9€Ȁ€琑K€‰€‚€€‚ÿu Sorting is the most throughly studied problem in computer science. Literally dozens of different algorithms are known, most of which possess some advantage over all other algorithms in certain situations. To become convinced of this, the reader is encouraged to browse through [Knu73b], with hundreds of pages of interesting sorting algorithms and analysis.u Most of the interesting ideas used in the design of algorithms appear in the context of sorting, such as divide-and-conquer, data structures, and randomized algorithms.,’BE' € €€€‚ÿ)éD>E& €€€‚ÿS+E‘E( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)>EºE& €€€‚ÿµf‘EoFO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ºE F1UÿÿÿÿÿÿÿÿOÿÿÿÿ FÄF$oFÄF" €€€ ÿH F G1…ÿÿÿÿÿÿÿÿP GJGΌApplications of Sorting>ÄFJG# €6€€‚ÿApplications of Sorting xq GÂI ܁󀀁çz8ۀ†"€ ‰€çgxÊ«€†"€ ‰€çx8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€濱Û/€†"€‰€æÀ±Û/€†"€‰€æÁ±Û/€†"€‰€€çz8ۀ‰€€çgxÊ«€‰€€çx8ۀ‰€‚€‚€‚‚ÿ Next: Approaches to Sorting Up: Data Structures and Sorting Previous: SortingApplications of Sorting An important key to algorithm design is to use sorting as a basic building block, because once a set of items is sorted, many other problems become easy. Consider the following applications:  0JGCLQ p€e€Ȁ€€€†"€W‚€€€€€†"€‚ÿu Searching - Binary search enables you to test whether an item is in a dictionary in time, once the keys are all sorted. Search preprocessing is perhaps the single most important application of sorting.u Closest pair - Given a set of n numbers, how do you find the pair of numbers that have the smallest difference between them? After the numbers are sorted, the closest pair of numbers will lie next to each other somewhere in sorted order. Thus a linear-time scan through them completes the job, for a total of time including the sorting.  É7ÂI’ ò€y€Ȁ€€€€‚€€€€€€€€€€€†"€ž€€€€€€†"€W†"€Ÿ†"€ †"€¡‚ÿu Element uniqueness - Are there any duplicates in a given a set of n items? The most efficient algorithm is to sort them and then do a linear scan though them checking all adjacent pairs. This is a special case of the closest-pair problem above, where we ask if there is a pair separated by a gap of zero.  u Frequency distribution - Given a set of n items, which element occurs the largest number of times in the set? If the items are sorted, we can sweep from left to right and count them, since all identical items will be lumped together during sorting. To find out how often an arbitrary element k occurs, start by looking up k using binary search in a sorted array of keys. By walking to the left of this point until the element is not k and then walking to the right, we can find this cCLÄFount in time, where c is the number of occurrences of k. The number of instances of k can be found in time by using binary search to look for the positions of both and , where is arbitrarily small, and then taking the difference of these positions.  šBCL²‚X ~€‡€Ȁ€€€€€€€€çÓh€†"€‰€€€‚ÿu Selection - What is the kth largest item in the set? If the keys are placed in sorted order in an array, the kth largest can be found in constant time by simply looking at the kth position of the array. In particular, the median element (see Section ) appears in the (n/2)nd position in sorted order.  }H/ƒ5 :€’€Ȁ†"€¢€€‚ÿ  Figure: Constructing the convex hull of points in the plane  ?ಂn…_ Œ€Å€Ȁ€€€€€çêÜìð€†"€‰€çʕ!€†"€‰€‚ÿu Convex hulls - Given n points in two dimensions, what is the polygon of smallest area that contains them all? The convex hull is like a rubber band stretched over the points in the plane and then released. It compresses to just cover the points, as shown in Figure . The convex hull gives a nice representation of the shape of the points and is the most important building block for more sophisticated geometric algorithms, as discussed in catalog Section .  d/ƒþ‡, &€É€Ȁ€€‚ÿBut how can we use sorting to construct the convex hull? Once you have the points sorted by x-coordinate, the points can be inserted from left to right into the hull. Since the rightmost point is always on the boundary, we know that it will be inserted into the hull. Adding this new rightmost point might cause others to be deleted, but we can quickly identify these points because they lie inside the polygon formed by adding the new point. These points to delete will be neighbors of the previous point we inserted, so they will be easy to find. The total time is linear after the sorting has been done. ån… Š( €Ë€€€‚ÿWhile some of these problems (particularly median and selection) can be solved in linear time using more sophisticated algorithms, sorting provides quick and easy solutions to all of these problems. It is a rare application whose time complexity is such that sorting proves to be the bottleneck, especially a bottleneck that could have otherwise been removed using more clever algorithmics. Don't ever be afraid to spend time sorting whenever you use an efficient sorting routine.)þ‡4Š& €€€‚ÿŒ» Šð‹ Ё‡€€‚çz8ۀ†"€ ‰€çgxÊ«€†"€ ‰€çx8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ±Û/€†"€‰€æñÛ/€†"€‰€æıÛ/€†"€‰€€çz8ۀ‰€€çgxÊ«€‰€€çx8ۀ‰€€‚‚ÿ Next: Approaches to Sorting Up: Data Structures and Sorting Previous: Sorting AlgorithmsMon Jun 2 23:33:50 EDT 1997)4ŠŒ& €€€‚ÿµfð‹ΌO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ŒÿŒ1UÿÿÿÿÿÿÿÿQÿÿÿÿÿŒ#$Ό#" €€€ ÿFÿŒi1ŠÿÿÿÿÿÿÿÿRi¥…ÃApproaches to Sorting<#¥# €2€€‚ÿApproaches to Sorting iÁÀ  ⁀€ç{8ۀ†"€ ‰€çgxÊ«€†"€ ‰€çy8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æB°Û/€†"€‰€æ9°Û/€†"€‰€æ:°Û/€†"€‰€€ç{8ۀ‰€€çgxÊ«€‰€€çy8ۀ‰€‚€‚€‚€‚ÿ Next: Data Structures Up: Data Structures and Sorting Previous: Applications of SortingApproaches to Sorting   Sorting is a natural laboratory for studying basic algorithm design paradigms, since many useful techniques lead to interesting sorting algorithms. Inde¥ÁÀ#ed, we introduce several techniques here that will be described further in subsequent chapters. Consider the algorithms below as case studies for each of the relevant techniques.)¥êÀ& €€€‚ÿ&ÁÀÁ# €€€‚ÿõ„êÀÂq °€ €Ȁç®ùù®€‰€‚€çˆ€‰€‚€çpm€‰€‚€çÑ€ ̀‰€‚€ç23+€‰€‚ÿu Data Structures u Incremental Insertion u Divide and Conquer u Randomization u Bucketing Techniques&Á+Â# €€€‚ÿ)ÂTÂ& €€€‚ÿS++§Â( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)TÂÐÂ& €€€‚ÿµf§Â…ÃO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1жÃ1UÿÿÿÿÿÿÿÿSÿÿÿÿ¶ÃÚÃ$…ÃÚÃ" €€€ ÿ@¶ÃÄ1O ÿÿÿÿÿÿÿÿTÄPÄnÎData Structures6ÚÃPÄ# €&€€‚ÿData Structures Ž¯ÄÇ ؁o€€ç|8ۀ†"€ ‰€çz8ۀ†"€ ‰€çz8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ;°Û/€†"€‰€æ<°Û/€†"€‰€æ=°Û/€†"€‰€€ç|8ۀ‰€€çz8ۀ‰€€çz8ۀ‰€‚€ ‚€‚ÿ Next: Incremental Insertion Up: Approaches to Sorting Previous: Approaches to SortingData StructuresPerhaps the most dramatic algorithmic improvement made possible by using appropriate data structures occurs in sorting. Selection sort is a simple-to-code algorithm that repeatedly extracts the smallest remaining element from the unsorted part of the set:  Â7PÄÆÉ‹ ä€q€€‚€€‚€€€€‚€€€€€€‚€€‚€€‚€€€€€€€†"€‚ÿ SelectionSort(A) For i = 1 to n do Sort[i] = Find-Minimum from A Delete-Minimum from A Return(Sort) Selection sort is typically implemented by partitioning the input array into sorted and unsorted regions. To find the smallest item, we perform a linear sweep through the unsorted portion of the array. The smallest item is then swapped with the ith item in the array before moving on the next iteration. Selection sort performs n iterations, where the average iteration takes n/2 steps, for a total of time.NçÇÍg œ€Õ€€€€€€€€†"€W€€€€†"€†"€€€€‚ÿBut what if we improve the data structure? It takes O(1) time to remove a particular item from an unsorted array once it has been located, but O(n) time to find the smallest item. These two are exactly the operations supported by priority queues. So what happens if we replace the data structure with a better priority queue implementation, either a heap or a balanced binary tree. Operations within the loop now take time each, instead of O(n). By using such a priority queue implementation, selection sort is sped up to from . The name typically given to this algorithm, heapsort, obscures the relationship between them, but heapsort is nothing but an implementation of selection sort with the right data structure.   )ÆÉ=Í& €€€‚ÿS+͐Í( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)=͹Í& €€€‚ÿµfÍnÎO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¹ÍŸÎ1UÿÿÿÿÿÿÿÿUÿÿÿÿŸÎÃÎ$nÎÃÎ" €€€ ÿFŸÎ Ï1‡ÿÿÿÿÿÿÿÿV ÏEύIncremental Insertion<ÃÎEÏ# €2€€‚ÿIncremental Insertion xs ÏÉ ؁÷€€ç}8ۀ†"€ ‰€çz8ۀ†"€ ‰€ç{8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ>°Û/€†"€‰€æ?°Û/€†"€‰€æ@°Û/EÏÉÃ΀†"€‰€€ç}8ۀ‰€€çz8ۀ‰€€ç{8ۀ‰€‚€ ‚€‚ÿ Next: Divide and Conquer Up: Approaches to Sorting Previous: Data StructuresIncremental InsertionNow consider a different approach to sorting that grows the sorted set one element at a time. Select an arbitrary element from the unsorted set, and put it in the proper position in the sorted set.   jµEÏ3µ 8o€€‚€€‚†"€£‚€€€€‚€€€€‚€€€€€€€€€€€€€€€€‚€†"€€€€€€€€‚ÿ InsertionSort(A) for i = 1 to n-1 do j=i while (A[j] > A[j-1]) do swap(A[j],A[j-1]) Although insertion sort takes in the worst case, it will perform considerably better if the data is almost sorted, since few iterations of the inner loop will suffice to sift it into the proper position. Insertion sort is perhaps the simplest example of the incremental insertion technique, where we build up a complicated structure on n items by first building it on n-1 items and then making the necessary changes to fix things in adding the last item. Incremental insertion proves a particularly useful technique in geometric algorithms.)É\& €€€‚ÿS+3¯( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)\Ø& €€€‚ÿµf¯O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ØŸ1UÿÿÿÿÿÿÿÿWÿÿÿÿŸâ$â" €€€ ÿCŸ%1HÿÿÿÿÿÿÿÿX%^uDivide and Conquer9â^# €,€€‚ÿDivide and Conquer ê³%H 7 <‚{€€ç~8ۀ†"€ ‰€çz8ۀ†"€ ‰€ç|8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æA°Û/€†"€‰€æm°Û/€†"€‰€æd°Û/€†"€‰€€ç~8ۀ‰€€çz8ۀ‰€€ç|8ۀ‰€‚€ ‚€‚€€€€€€†"€€†"€¥€€€€‚ÿ Next: Randomization Up: Approaches to Sorting Previous: Incremental InsertionDivide and Conquer Suppose we take the n elements to sort and split them into piles S and T, each with half the elements. After sorting both piles, it is easy to combine the two sorted piles. To merge and , note that the smallest item must sit at the top of one of the two lists. Once identified, the smallest element can be removed, and the second smallest item will again be atop one of the two lists. Repeating this operation merges the two sorted lists in O(n) time. This provides the basis for a recursive algorithm.   Ój^i  €Ý€€‚€€€€‚†"€Š†"€§‚€†"€š†"€‚€‚ÿ Mergesort(A[1,n]) Merge( MergeSort( ), MergeSort( ) ) Because the recursion goes levels deep and a linear amount of work is done per level, Mergesort takes time in the worst case.Mergesort is a classic divide-and-conquer algorithm. Whenever we can break one large problem into two smaller problems, we are ahead of the game because the smaller problems are easier. The trick is taking advantage of the two partial solutions to put together a solution of the full problem. The merge operation takes care of the reassembly in mergesort, but not all problems can be so neatly decomposed.)H D& €€€‚ÿS+—( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)DÀ& €€€‚ÿµf—uO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÀŠ1UÿÿÿÿÿÿÿÿYÿÿÿÿŠ @$u @" €€€ ÿŠ @u> ŠJ@1ÑÿÿÿÿÿÿÿÿZJ@~@ê€Randomization4 @~@# €"€€‚ÿRandomization Œ]J@ D/ ,‚Ë€€ç8ۀ†"€ ‰€çz8ۀ†"€ ‰€ç}8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æe°Û/€†"€‰€æf°Û/€†"€‰€æg°Û/€†"€‰€€ç8ۀ‰€€çz8ۀ‰€€ç}8ۀ‰€‚€ ‚€€€€€€€€€€€€€€€‚ÿ Next: Bucketing Techniques Up: Approaches to Sorting Previous: Divide and ConquerRandomizationSuppose we select, at random, one of the n items we seek to sort. Call this element p. In quicksort, we separate the n-1 other items into two piles: a low pile containing all the elements that will appear before p in sorted order and a high pile containing all the elements that will appear after p in sorted order. After sorting the two piles and arranging the piles correctly, we have succeeded in sorting the n items.   9a~@CFØ ~Å€€‚€€‚€€€€‚€€€€‚€€‚€€‚‚€€‚€€€€€€†"€©‚€€€€€€‚€€€€‚€€‚€€€€€€‚ÿ Quicksort(A, low, high) if (low < high) ploc = Partition(A,low,high) Quicksort(A,low, ploc - 1) Quicksort(A, ploc+1, high) Partition(A,low,high) swap(A[low],A[random( )]) pivot = A[low] leftwall = low for i = low+1 to high if (A[i] < pivot) thenÍB DJ‹ 䀋€€€€€€‚€€€€€€€€‚€€€€€€€€‚€†"€†"€†"€‚ÿ leftwall = leftwall+1 swap(A[i],A[leftwall]) swap(A[low],A[leftwall]) Mergesort ran in time because we split the keys into two equal halves, sorted them recursively, and then merged the halves in linear time. Thus whenever our pivot element is near the center of the sorted array (i.e. the pivot is close to the median element), we get a good split and realize the same performance as mergesort. Such good pivot elements will often be the case. After all, half the elements lie closer to the middle than one of the ends. On average, Quicksort runs in time. If we are extremely unlucky and our randomly selected elements always are among the largest or smallest element in the array, Quicksort turn into selection sort and runs in . However, the odds against this are vanishingly small.V1CFfK% €c€€‚ÿRandomization is a powerful, general tool to improve algorithms with bad worst-case but good average-case complexity. The worst case examples still exist, but they depend only upon how unlucky we are, not on the order that the input data is presented to us. For randomly chosen pivots, we can say that|OJâK- *€ €Ȁ†"€ª‚ÿ``Randomized quicksort runs in time on any input, with high probability.''֟fKžL7 <€?€€€€€€€€‚ÿIf instead, we used some deterministic rule for selecting pivots (like always selecting A[(low+high)/2] as pivot), we can make no claim stronger than…XâK=M- *€²€Ȁ†"€ª‚ÿ``Quicksort runs in time if you give me random input data, with high probability.''ŸžLûM= H€€€ç Æ•ù€†"€‰€€‚ÿRandomization can also be used to drive search techniques such as simulated annealing, which are discussed in Section .)=M$N& €€€‚ÿÀ¿ûM € Ё€€‚ç8ۀ†"€ ‰€çz8ۀ†"€ ‰€ç}8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æh°Û/€†"€‰€æi°Û/€†"€‰€æj°Û/€†"€‰€€ç8ۀ‰€€çz8ۀ‰€€ç}8ۀ‰€€‚‚ÿ Next: Bucketing Techniques Up: Approaches to Sorting Previous: Divide and Conquer AlgorithmsMon Jun 2 23:33:50 EDT 1997$N € @)$N5€& €€€‚ÿµf €ê€O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b15€1Uÿÿÿÿÿÿÿÿ[ÿÿÿÿ?$ê€?" €€€ ÿE„1Ý ÿÿÿÿÿÿÿÿ\„¿kBucketing Techniques;?¿# €0€€‚ÿBucketing Techniques yb„8… üÕ€€ç«8ۀ†"€ ‰€çz8ۀ†"€ ‰€ç~8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æk°Û/€†"€‰€æl°Û/€†"€‰€昰Û/€†"€‰€€ç«8ۀ‰€€çz8ۀ‰€€ç~8ۀ‰€‚€ ‚€€€€€€€‚ÿ Next: War Story: Stripping Triangulations Up: Approaches to Sorting Previous: RandomizationBucketing TechniquesIf we were sorting names for the telephone book, we could start by partitioning the names according to the first letter of the last name. That will create 26 different piles, or buckets, of names. Observe that any name in the J pile must occur after every name in the I pile but before any name in the K pile. Therefore, we can proceed to sort each pile individually and just concatenate the bunch of piles together.   ï¿T‰- (€ß€€‚€€‚ÿIf the names are distributed fairly evenly among the buckets, as we might expect, the resulting 26 sorting problems should each be substantially smaller than the original problem. Further, by now partitioning each pile based on the second letter of each name, we generate smaller and smaller piles. The names will be sorted as soon as each bucket contains only a single name. The resulting algorithm is commonly called bucketsort or distribution sort.Bucketing is a very effective idea whenever we are confident that the distribution of data will be roughly uniform. It is the idea that underlies hash tables, kd-trees, and a variety of other practical data structures. The downside of such techniques is that the performance can be terrible whenever the data distribution is not what we expected. Although data structures such as binary trees offer guaranteed worst-case behavior for any input distribution, no such promise exists for heuristic data structures on unexpected input distributions.   FÊ8…šŒ| ƀ›€€†"€«€€‚ç.3+€†"€‰€€€€€€€€€€€†"€¬€€€‚ÿ  Figure: A small subset of Charlottesville Shiffletts  To show that non-uniform distributions occur in real life, consider Americans with the uncommon last name of Shifflett. The 1997 Manhattan telephone directory, with over one million names, contains exactly five Shiffletts. So how many Shiffletts should there be in a small city of 50,000 people? Figure shows a small portion of the two and a half pages of Shiffletts in the Charlottesville, Virginia telephone book. The Shifflett clan is a fixture of the region, but it would play havoc with any distribution sort program, as refining buckets from S to Sh to Shi to Shif to to Shifflett results in no significant partitioning.)T‰Ì& €€€‚ÿÊɚŒŽ Ё£€€‚ç«8ۀ†"€ ‰€çz8ۀ†"€ ‰€ç~8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€揰Û/€†"€‰€搰Û/€†"€‰€摰Û/€†"€‰€€ç«8ۀ‰€€çz8ۀ‰€€ç~8ۀ‰€€‚‚ÿ Next: War Story: Stripping Triangulations Up: Approaches to Sorting Previous: Randomization AlgorithmsMon Jun 2 23:33:50 EDT 1997)̶Ž& €€€‚ÿµfŽkO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¶Žœ1Uÿÿÿÿÿÿÿÿ]ÿÿÿÿœÀ$kÀ" €€€ ÿT#œ À1—,ÿÿÿÿÿÿÿÿ^ ÀjÀÁFWar Story: StriÀ ÀÀpping TriangulationsJ'ÀjÀ# €N€€‚ÿWar Story: Stripping Triangulations C, À­Â ük€€ç¢8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€撰Û/€†"€‰€擰Û/€†"€‰€攰Û/€†"€‰€€ç¢8ۀ‰€€çgxÊ«€‰€€ç8ۀ‰€‚€‚€‚†"€­€€‚ÿ Next: War Story: Mystery of Up: Data Structures and Sorting Previous: Bucketing TechniquesWar Story: Stripping Triangulations     Figure: (a) A triangulated model of a dinosaur (b) Several triangle strips in the model  ºHjÀgÇr ²€—€€çØ«4€€†"€‰€‚€€çØ«4€€†"€‰€€€çÑÜ؈€†"€‰€‚ÿThe most common type of geometric model used for computer graphics describes the geometry of the object as a triangulated surface, as shown in Figure a. High-performance rendering engines have special hardware for rendering and shading triangles. This hardware is so fast that the bottleneck of rendering is simply feeding the structure of the triangulation into the hardware engine.     Although each triangle can be described by specifying its three endpoints and any associated shading/normal information, an alternative representation is more efficient. Instead of specifying each triangle in isolation, suppose that we partition the triangles into strips of adjacent triangles and walk along the strip, as shown in Figure (b). Since each triangle shares two vertices in common with its neighbors, we save the cost of retransmitting the two extra vertices and any associated normals. To make the description of the triangles unambiguous, the Silicon Graphics triangular-mesh renderer OpenGL assumes that all turns alternate from left to right (as shown in Figure ).  ,¿­Â“Êm š€…€€†"€®€€‚€€€€皧œ€†"€‰€ç-€†"€‰€‚ÿ  Figure: Partitioning a triangular mesh into strips: (a) with left-right turns (b) with the flexibility of arbitrary turns  The problem of finding a small number of strips that cover each triangle in a mesh can be thought of as a graph problem, where this graph has a vertex for every triangle of the mesh, and there is an edge between every pair of vertices representing adjacent triangles. This dual graph representation of the planar subdivision representing the triangulation (see Section ) captures all the information about the triangulation needed to partition it into triangle strips. Section describes our experiences constructing the graph from the triangulation.    ÐgǝÌ: B€£€€ç$n‚Š€†"€‰€‚ÿOnce we had the dual graph available, the project could begin in earnest. We sought to partition the vertices of the dual graph into as few paths or strips as possible. Partitioning it into one path implied that we had discovered a Hamiltonian path, which by definition visits each vertex exactly once. Since finding a Hamiltonian path was NP-complete (see Section ), we knew not to look for an optimal algorithm, but to concentrate instead on heuristics.a<“ÊþÎ% €y€€‚ÿIt is always best to start with simple heuristics before trying more complicated ones, because simple might well suffice for the job. The most natural heuristic for strip cover would be to start from an arbitrary triangle and then do a left-right walk from there until the walk ends, either by hitting the boundary of the object or a previously visited triangle. This heuristic had the advantage that it would be fast and simple, although there could be no reason to suspect that it should find the smallest possible set of left-right strips for a given triangulation.òÌ)- (€å€€€€‚‚ÿA heuristic more likely to result in a small number of strips would be greedy. Greedy heuristics always try to grab the best possible thing first. In the case of the triangulation, the natural greedy heuristicþÎ)À would find the starting triangle that yields the longest left-right strip, and peel that one off first.  Being greedy also does not guarantee you the best possible solution, since the first strip you peel off might break apart a lot of potential strips we would have wanted to use later. Still, being greedy is a good rule of thumb if you want to get rich. Since removing the longest strip would leave the fewest number of triangles for later strips, it seemed reasonable that the greedy heuristic would out-perform the naive heuristic.0áþÎYO l€Ç€€€€€€†"€¯€€€€†"€‚ÿBut how much time does it take to find the largest strip to peel off next? Let k be the length of the walk possible from an average vertex. Using the simplest possible implementation, we could walk from each of the n vertices per iteration in order to find the largest remaining strip to report in time. With the total number of strips roughly equal to n/k, this yields an -time implementation, which would be hopelessly slow on a typical model of 20,000 triangles.òÍ)K% €›€€‚ÿHow could we speed this up? It seems wasteful to rewalk from each triangle after deleting a single strip. We could maintain the lengths of all the possible future strips in a data structure. However, whenever we peel off a strip, we would have to update the lengths of all the other strips that will be affected. These strips will be shortened because they walked through a triangle that now no longer exists. There are two aspects of such a data structure:}HYÈ5 :€’€Ȁ†"€°€€‚ÿ  Figure: A bounded height priority queue for triangle strips  &ÍKî Y €€Ÿ€Ȁ€€€çÌÜ؈€†"€‰€ç|þ4.€†"€‰€‚ÿu Priority Queue - Since we were repeatedly interested in identifying the longest possible next strip, we needed a priority queue to store the strips, ordered according to length. The next strip to peel would always be the top of the queue. Our priority queue had to permit reducing the priority of arbitrary elements of the queue whenever we updated the strip lengths to reflect what triangles were peeled away. Because all of the strip lengths were bounded by a fairly small integer (hardware constraints prevent any strip from having more than 256 vertices), we used a bounded height priority queue (shown in Figure and described in Section ). An ordinary heap would also have worked just fine.  U%ÈC 0 .€K€Ȁ‚€€€‚ÿTo update a queue entry associated with a triangle, we needed to be able to quickly find where it was. This meant that we also needed a ...u Dictionary - For each triangle in the mesh, we needed a way to find where it was in the queue. This meant storing a pointer for each triangle. Since each triangle was defined by three integer vertex numbers, either a hash table or an array of lists of triangles would suffice. By integrating this dictionary with the priority queue, we built a data structure capable of a wider range of operations.  }/î ÀN j€c€€‚†"€±€€‚çÐÜ؈€†"€‰€‚ÿAlthough there were various other complications, such as quickly recalculating the length of the strips affected by the peeling, the key idea needed to obtain better performance was to use the priority queue. Run time improved by several orders of magnitude after employing these data structures.  Figure: A comparison of the naive versus greedy heuristics for several triangular meshes  How much better did the greedy heuristic do than the naive heuristic? Consider the table in Figure . In all cases, the greedy heuristic led to a set of strips that cost less, as measured by the total size of the strips. The savings ranged from about 10% to 50%, quite remarkable, since the greatest possible improvement (going from three vertices per triangle down to one) could yield a savings of only 66.6%.%ÑC ñCT v€§€€†"€²€€€€‚†"€ÀñCÀç‚K€‰€€‚ÿAfter implementing the greedy heuristic with our priority queue data structure, our complete algorithm ran in time, where n is the number of triangles and k is the length of the average strip. Thus the torus, which consisted of a small number of very long strips, took longer than the jaw, even though the latter contained over three times as many triangles.There are several lessons to be gleaned from this story. First, whenever we are working with a large enough data set, only linear or close to linear algorithms (say ) are likely to be fast enough. Second, choosing the right data structure is often the key to getting the time complexity down to this point. Finally, using a greedy or somewhat smarter heuristic over the naive approach is likely to significantly improve the quality of the results. How much the improvement is likely to be can be determined only by experimentation. Our final, optimized triangle strip program is described in [ESV96].)ÀD& €€€‚ÿÉÈñCãE Ё¡€€‚ç¢8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€数Û/€†"€‰€新Û/€†"€‰€旰Û/€†"€‰€€ç¢8ۀ‰€€çgxÊ«€‰€€ç8ۀ‰€€‚‚ÿ Next: War Story: Mystery of Up: Data Structures and Sorting Previous: Bucketing Techniques AlgorithmsMon Jun 2 23:33:50 EDT 1997)D F& €€€‚ÿµfãEÁFO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 FòF1Uÿÿÿÿÿÿÿÿ_ÿÿÿÿòFG$ÁFG" €€€ ÿS"òFiG1=,ÿÿÿÿÿÿÿÿ`iG²G¬ÏWar Story: Mystery of the PyramidsI&G²G# €L€€‚ÿWar Story: Mystery of the Pyramids *#iGÜI ܁W€€ç£8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç«8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æðÛ/€†"€‰€溰Û/€†"€‰€æ»°Û/€†"€‰€€ç£8ۀ‰€€çgxÊ«€‰€€ç«8ۀ‰€‚€‚€‚‚ÿ Next: War Story: String 'em Up: Data Structures and Sorting Previous: War Story: Stripping TriangulationsWar Story: Mystery of the Pyramids That look in his eyes should have warned me even before he started talking.K"²G'L) €E€€‚‚‚ÿ``We want to use a parallel supercomputer for a numerical calculation up to 1,000,000,000, but we need a faster algorithm to do it.''  I'd seen that distant look before. Eyes dulled from too much exposure to the raw horsepower of supercomputers. Machines so fast that brute force seemed to eliminate the need for clever algorithms. So it always seemed, at least until the problems got hard enough.``I am working with a Nobel prize winner to use a computer on a famous problem in number theory. Are you familiar with Waring's problem?''   w#ÜIžOT v€M€€çA‘K€‰€†"€³‚€€†"€Ž†"€µ‚ÿI knew some number theory [NZ80]. ``Sure. Waring's problem asks whether every integer can be expressed at least one way as the sum of at most four integer squares. For example, . It's a cute problem. I remember proving that four squares suffice in my undergraduate number theory class. Yes, it's a famous problem, sure, but one that got solved about 200 years ago.''``No, we are interested in a different version of Waring's problem. A pyramidal number is a number of the form , for . Thus the first several pyramidal numbers are 1, 4, 10, 20, 35, 56, 84, 120, and 165. The conjecture since 1928 is that every integer can be represented by the sum of at most five such pyramidal numbers. We want to use a supercomputer to prove this conjecture on all numbers from 1 to 1,000,000,000.'' æ'L·' €Í€€‚‚ÿ``Doing a billion of anything will take a serious amount žO·Gof time,'' I warned. ``The time you spend to compute the minimum representation of each number will be critical, because you are going to do it one billion times. Have you thought about what kind of an algorithm you are going to use?''``We have already written our program and run it on a parallel supercomputer. It works very fast on smaller numbers. Still, it take much too much time as soon as we get to 100,000 or so.'''øžOރ/ ,€ñ€€€€‚‚‚ÿ``Terrific,'' I thought. Our supercomputer junkie had discovered asymptotic growth. No doubt his algorithm ran in something like quadratic time, and he got burned as soon as n got large.``We need a faster program in order to get to one billion. Can you help us? Of course, we can run it on our parallel supercomputer when you are ready.''I'll confess that I am a sucker for this kind of challenge, finding better algorithms to speed up programs. I agreed to think about it and got down to work.'Ò·ˆU x€©€€†"€¶€€€€€€€€†"€·€€‚ÿI started by looking at the program that the other guy had written. He had built an array of all the pyramidal numbers from 1 to n. To test each number k in this range, he did a brute force test to establish whether it was the sum of two pyramidal numbers. If not, the program tested whether it was the sum of three of them, then four, and finally five of them, until it first got an answer. About 45% of the integers are expressible as the sum of three pyramidal numbers, while most of the remaining 55% require the sum of four; usually each can be represented many different ways. Only 241 integers are known to require the sum of five pyramidal numbers, the largest one being 343,867. For about half of the n numbers, this algorithm presumably went through all of the three-tests and at least some of the four-tests before terminating. Thus the total time for this algorithm would be at least time, where n= 1,000,000,000. No wonder his program cried uncle.9áރ>ŠX ~€Å€€€€€€€€çâ .I€†"€‰€€€€€‚ÿAnything that was going to do significantly better on a problem this large had to avoid explicitly testing all triples. For each value of k, we were seeking the smallest number of pyramidal numbers that sum exactly to k. This problem has a name, the knapsack problem, and it is discussed in Section . In our case of interest, the weights are the set of pyramidal numbers no greater than n, with an additional constraint that the knapsack holds exactly k items.  ¡_ˆߍB R€Á€€€€‚€€†"€ž€€‚ÿThe standard approach to solving knapsack precomputes the sum of smaller subsets of the items for use in computing larger subsets. In other words, if we want to know whether k is expressible as the sum of three numbers, and we have a table of all sums of two numbers, all we have to do is ask whether our number is expressible as the sum of a single number plus a number in this two-table.Thus what I needed was a table of all integers less than n that could be expressed as the sum of two of the 1,818 pyramidal numbers less than 1,000,000,000. There could be at most = 3,305,124 of them. Actually, there would be only about half this many because we could eliminate duplicates, or any sum bigger than our target. Building a data structure, such as a sorted array, to hold these numbers would be no big deal. Call this data structure the two-table.N>Šë² 2§€€€€€€€€€€€€€€€€†"€¹€€€€€€€€†"€º€€€€†"€»€€†"€Œ†"€€€€€‚ÿTo find the minimum decomposition for a given k, I would start out by checking whether it was one of the 1,818 pyramidal numbers. If not, I would then search to see whether k was in the sorted table of the sums of two pyramidal numbers. If it wasn't, to see whether k was expressible as the sum of three such numbers, all I had to do was check whether k - ߍëÂGp[i] was in the two-table for . This could be done quickly using binary search. To see whether k was expressible as the sum of four pyramidal numbers, I had to check whether k - two[i] was in the two-table for all . However, since almost every k was expressible in many ways as the sum of four pyramidal numbers, this latter test would terminate quickly, and the time taken would be dominated by the cost of the threes. Testing whether k was the sum of three pyramidal numbers would take . Running this on each of the n integers gives an algorithm for the complete job. Comparing this to his algorithm for n= 1,000,000,000 suggested that my algorithm was a cool 30,000 times faster than the original!g!ߍRÅF Z€E€€€€çfZHŀ†"€‰€€€‚ÿMy first attempt to code this up solved up to n=1,000,000 on my crummy Sparc ELC in about 20 minutes. From here, I experimented with different data structures to represent the sets of numbers and different algorithms to search these tables. I tried using hash tables and bit vectors instead of sorted arrays and experimented with variants of binary search such as interpolation search (see Section ). My reward for this work was solving up to n=1,000,000 in under three minutes, a factor of six improvement over my original program.ßëÂbÇ1 0€¿€€€€€€‚ÿWith the real thinking done, I worked to tweak a little more performance out of the program. I avoided doing a sum-of-four computation on any k when k-1 was the sum-of-three, since 1 is a pyramidal number, saving about 10% of the total run time on this trick alone. Finally, I got out my profiler and tried some low-level tricks to squeeze a little more performance out of the code. For example, I saved another 10% by replacing a single procedure call with in-line code.ÓRÅqÊ< F€©€€çÇIÝ*€†"€‰€‚‚ÿAt this point, I turned the code over to the supercomputer guy. What he did with it is a depressing tale, which is reported in Section .In writing up this war story, I went back to rerun our program almost five years later. On my desktop Sparc 5, getting to 1,000,000 now took 167.5 seconds using the cc compiler without turning on any compiler optimization. With level 2 optimization, the job ran in only 81.8 seconds, quite a tribute to the quality of the optimizer. The gcc compiler with optimization did even better, taking only 74.9 seconds to get to 1,000,000. The run time on my desktop machine had improved by a factor of about three over a four-year period. This is probably typical for most desktops.  \4bÇÍÌ( €i€€€‚ÿThe primary importance of this war story is to illustrate the enormous potential for algorithmic speedups, as opposed to the fairly limited speedup obtainable via more expensive hardware. I sped his program up by about 30,000 times. His million-dollar computer had 16 processors, each reportedly five times faster on integer computations than the $3,000 machine on my desk. That gave a maximum potential speedup of less than 100 times. Clearly, the algorithmic improvement was the big winner here, as it is certain to be in any sufficiently large computation.)qÊöÌ& €€€‚ÿØ×ÍÌÎÎ Ё¿€€‚ç£8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç«8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€挰Û/€†"€‰€朰Û/€†"€‰€柰Û/€†"€‰€€ç£8ۀ‰€€çgxÊ«€‰€€ç«8ۀ‰€€‚‚ÿ Next: War Story: String 'em Up: Data Structures and Sorting Previous: War Story: Stripping Triangulations AlgorithmsMon Jun 2 23:33:50 EDT 1997)öÌ÷Î& €€€‚ÿµfÎάÏO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1÷ÎÝÏ1UÿÿÿÿÿÿÿÿaÿÿÿÿÝÏ $¬Ï " €€€ ÿÝÏ ¬ÏIÝÏV1Ó4ÿÿÿÿÿÿÿÿbV•ÙÂWar Story: String 'em Up? •# €8€€‚ÿWar Story: String 'em Up ÑÊVf ܁¥€€ç€8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç¢8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ¿°Û/€†"€‰€æÀ°Û/€†"€‰€æÁ°Û/€†"€‰€€ç€8ۀ‰€€çgxÊ«€‰€€ç¢8ۀ‰€‚€‚€‚‚ÿ Next: Exercises Up: Data Structures and Sorting Previous: War Story: Mystery ofWar Story: String 'em Up Biologists are hard at work on a fifteen-year project to sequence the human genome. This pattern of nucleotides encodes all the information necessary to build the proteins that we are built of. This project has already had an enormous impact on medicine and molecular biology.  ‚^•è$ €Œ€€‚ÿAlgorists have become interested in the human genome project as well, for several reasons: §fòc ”€S€Ȁ€€ç{ÜŠ€†"€‰€çÁ2>à€†"€‰€‚€€€€‚ÿu DNA sequences can be accurately represented as strings of characters on the four-letter alphabet (A,C,T,G). Biologist's needs have sparked new interest in old algorithmic problems (such as string matching - see Section ) as well as creating new problems of substantial interest (such as shortest common superstring - see Section ).  u DNA sequences are very long strings. The human genome is approximately three billion base pairs (or characters) long. Thus sophisticated computational techniques are necessary to deal with them. Such large problem sizes means that asymptotic (big-Oh) complexity analysis is usually fully justified on biological problems.¬„èž( € €Ȁ€‚ÿu Enough money is being invested in the human genome project for computer scientists to want to claim their piece of the action.v3ò C T€g€€ç"ƒK€‰€ç͘K€‰€€€‚ÿMy particular interest in computational biology has revolved around a recently proposed but algorithmically intensive technique for DNA sequencing called sequencing by hybridization (SBH) [CK94, PL94]. The traditional sequencing by hybridization procedure attaches a set of probes to an array, forming a sequencing chip. Each of these probes determines whether or not the probe string occurs as a substring of the DNA target. The target DNA can now be sequenced based on the constraints of which strings are and are not substrings of the target.    |Ež 7 <€€€†"€œ†"€Ÿ‚ÿOne problem with SBH is that enormous arrays (say strings) are necessary to sequence relatively short pieces of DNA (typically about 200 base pairs long). The major reason is that all of these probes are made at the same time. If you want to look up a name in the telephone book but are only allowed to consult the book once, you must copy down every single name from the book at that time. But if you are allowed to ask ``is the name before Mendoza?'' and wait for the answer before asking your next question, you can use binary search to greatly reduce your total effort.Œ3 XA‰ à€g€€‚‚€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿWe were convinced that using several small arrays would be more efficient than using one big array. We even had theory to justify our technique, but biologists aren't very inclined to believe theory. They demand experiments for proof. Hence we had to implement our algorithms and use simulation to prove that they worked.So much for motivation. The rest of this tale will demonstrate the impact that clever data structures can have on a string processing application.Our technique involved identifying all the strings of length 2k that are possible substrings of an unknown string S, given that we know all length k substrings of S. For example, suppose we know that AC, CA, and CC are the only length-2 substrings of S.  XA It is certainly possible that ACCA is a substring of S, since the center substring is one of our possibilities. However, CAAC cannot be a substring of S, since we know that AA is not a substring of S. We needed to find a fast algorithm to construct all the consistent length-2k strings, since S could be very long.O nDÇ \¥€€†"€¿€€€€‚€€†"€€€€€€€çÔ8ˆ€†"€‰€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿ  Figure: The concatentation of two fragments can be in S only if all subfragments are  The simplest algorithm to build the 2k strings would be to concatenate all pairs of k-strings together, and then for each pair to make sure that all (k-1) length-k substrings spanning the boundary of the concatenation were in fact substrings, as shown in Figure . For example, the nine possible concatenations of AC, CA, and CC are ACAC, ACCA, ACCC, CAAC, CACA, CACC, CCAC, CCCA, and CCCC. Only CAAC can be eliminated because of the absence of AA.œMXA GO l€Ÿ€€€€€€†"€W€€€€†"€À‚ÿWe needed a fast way of testing whether each of the k-1 substrings straddling the concatenation was a member of our dictionary of permissible k-strings. The time it takes to do this depends upon which kind of data structure we use to maintain this dictionary. With a binary search tree, we could find the correct string within comparisons, where each comparison involved testing which of two length-k strings appeared first in alphabetical order. Since each such comparison could require testing k pairs of characters, the total time using a binary search tree would be .  ÞnDJ) €œ€€‚‚‚ÿThat seemed pretty good. So my graduate student Dimitris Margaritis implemented a binary search tree data structure for our implementation. It worked great up until the moment we ran it.``I've tried the fastest computer in our department, but our program is too slow,'' Dimitris complained. ``It takes forever on strings of length only 2,000 characters. We will never get up to 50,000.''For interactive SBH to be competitive as a sequencing method, we had to be able to sequence long fragments of DNA, ideally over 50 kilobases in length. If we couldn't speed up the program, we would be in the embarrassing position of having a biological technique invented by computer scientists fail because the computations took too long.' G’LZ ‚€U€€†"€€€‚€€€€€€†"€Á†"€‚ÿWe profiled our program and discovered that almost all the time was spent searching in this data structure, which was no surprise. For each of the possible concatenations, we did this k-1 times. We needed a faster dictionary data structure, since search was the innermost operation in such a deep loop.``What about using a hash table?'' I suggested. ``If we do it right, it should take O(k) time to hash a k-character string and look it up in our table. That should knock off a factor of , which will mean something when 2,000.''  4JÆN2 2€€€‚‚†"€ÂÿDimitris went back and implemented a hash table implementation for our dictionary. Again, it worked great up until the moment we ran it.``Our program is still too slow,'' Dimitris complained. ``Sure, it is now about ten times faster on strings of length 2,000. So now we can get up to about 4,000 characters. Big deal. We will never get up to 50,000.''``We should have expected only a factor ten speedup,'' I mused. ``After all, . We need a faster data structure to search in our dictionary of strings.''Ïr’L¡] ˆ€å€€€€€€€€€€‚€€€€€€€€€€‚ÿ``But what can be faster than a hash table?'' Dimitris countered. ``To look up a k-character string, you must read all k characters. Our hash table already does O(k) searching.''``Sure, it takes k comparisonÆN¡ s to test the first substring. But maybe we can do better on the second test. Remember where our dictionary queries are coming from. When we concatenate ABCD with EFGH, we are first testing whether BCDE is in the dictionary, then CDEF. These strings differ from each other by only one character. We should be able to exploit this so that each subsequent test takes constant time to perform....''ýjÆNž…“ ô€Û€€€€€€‚†"€Ā€€€€€€€‚€€†"€ŀ€çqŠ ؀†"€‰€€€€€€€‚ÿ``We can't do that with a hash table,'' Dimitris observed. ``The second key is not going to be anywhere near the first in the table. A binary search tree won't help, either. Since the keys ABCD and BCDE differ according to the first character, the two strings will be in different parts of the tree.''  Figure: Suffix tree on ACAC and CACT, with the pointer to the suffix of ACAC  ``But we can use a suffix tree to do this,'' I countered. ``A suffix tree is a trie containing all the suffixes of a given set of strings. For example, the suffixes of ACAC are . Coupled with suffixes of string CACT, we get the suffix tree of Figure . By following a pointer from ACAC to its longest proper suffix CAC, we get to the right place to test whether CACT is in our set of strings. One character comparison is all we need to do from there.''  Ò¡pˆQ p€€€çýùր†"€‰€‚€€€€†"€ƂÿSuffix trees are amazing data structures, discussed in considerably more detail in Section . Dimitris did some reading about them, then built a nice suffix tree implementation for our dictionary. Once again, it worked great up until the moment we ran it.``Now our program is faster, but it runs out of memory,'' Dimitris complained. ``And this on a 128 megabyte machine with 400 megabytes virtual memory! The suffix tree builds a path of length k for each suffix of length k, so all told there can be nodes in the tree. It crashes when we go beyond 2,000 characters. We will never get up to strings with 50,000 characters.''ýž…mŒ~ ʀ€€çýùր†"€‰€‚€€çtŠ ؀†"€‰€€€ç‘K€‰€ç-Ðl€†"€‰€‚ÿI wasn't yet ready to give up. ``There is a way around the space problem, by using compressed suffix trees,'' I recalled. ``Instead of explicitly representing long paths of character nodes, we can refer back to the original string.'' Compressed suffix trees always take linear space, as described in Section .Dimitris went back one last time and implemented the compressed suffix tree data structure. Now it worked great! As shown in Figure , we ran our simulation for strings of length n= 65,536 on a SPARCstation 20 without incident. Our results, reported in [MS95a], showed that interactive SBH could be a very efficient sequencing technique. Based on these simulations, we were able to arouse interest in our technique from biologists. Making the actual wet laboratory experiments feasible provided another computational challenge, which is reported in Section .Épˆp: B€•€€çtŠ ؀†"€‰€‚ÿThe take home lessons for programmers from Figure should be apparent. We isolated a single operation (dictionary string search) that was being performed repeatedly and optimized the data structure we used to support it. We started with a simple implementation (binary search trees) in the hopes that it would suffice, and then used profiling to reveal the trouble when it didn't. When an improved dictionary structure still did not suffice, we looked deeper into what kinds of queries we were performing, so that we could identify an even better data structure. Finally, we didn't give up until we had achieved the level of performance we needed. In algorithms, as in life, persistence usually pays off.˜amŒÀ7 >€Ä€€†"€ǀ€€‚ÿ  Figure: Run times (in seconds) for the SBH simulation using various data structupÀ res  )p=À& €€€‚ÿŸœÀûÁ Ё‹€€‚ç€8ۀ†"€ ‰€çgxÊ«€†"€ ‰€ç¢8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ°Û/€†"€‰€æî°Û/€†"€‰€æå°Û/€†"€‰€€ç€8ۀ‰€€çgxÊ«€‰€€ç¢8ۀ‰€€‚‚ÿ Next: Exercises Up: Data Structures and Sorting Previous: War Story: Mystery of AlgorithmsMon Jun 2 23:33:50 EDT 1997)=À$Â& €€€‚ÿµfûÁÙÂO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1$ Ã1Uÿÿÿÿÿÿÿÿcÿÿÿÿ Ã.Ã$ÙÂ.Ã" €€€ ÿ: ÃhÃ1t*ÿÿÿÿÿÿÿÿdhØÃGMExercises0 .ØÃ# €€€‚ÿExercises ž³hÃPÅ ؁w€€çln_ɀ†"€ ‰€çgxÊ«€†"€ ‰€ç£8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€ææ°Û/€†"€‰€æç°Û/€†"€‰€æè°Û/€†"€‰€€çln_ɀ‰€€çgxÊ«€‰€€ç£8ۀ‰€‚€‚€‚ÿ Next: Implementation Challenges Up: Data Structures and Sorting Previous: War Story: String 'emExercises ºX˜Ã Éb ’€µ€Ȁ€€€€€€€€€€†"€‚€€†"€€€‚ÿ1. Newt Gingrich is given the job of partitioning 2n players into two teams of n players each. Each player has a numerical rating that measures how good he/she is at the game. Newt seeks to divide the players as unfairly as possible, so as to create the biggest possible talent imbalance between team A and team B. Show how Newt can do the job in time.  2. Take as input a sequence of 2n real numbers. Design an algorithm that partitions the numbers into n pairs, with the property that the partition minimizes the maximum sum of a pair. For example, say we are given the numbers (1,3,5,9). The possible partitions are ((1,3),(5,9)), ((1,5),(3,9)), and ((1,9),(3,5)). The pair sums for these partitions are (4,14), (6,12), and (10,8). Thus the third partition has 10 as its maximum sum, which is the minimum over the three partitions.šqPŲÊ7 <€ã€Ȁ€€€€€€‚ÿ3. Assume that we are given as input n pairs of items, where the first item is a number and the second item is one of three colors (red, blue, or yellow). Further, assume that the items are sorted by number. Give an O(n) algorithm to sort the items by color (all reds before all blues before all yellows) such that the numbers for identical colors stay sorted.3 ÉåË- (€ €Ȁ‚€€‚ÿFor example: (1,blue), (3,red), (4,blue), (6,yellow), (9,red) should become (3,red), (9,red), (1,blue), (4,blue), (6,yellow).4. (*) The mode of a set of numbers is the number that occurs most frequently in the set. The set (4,6,2,4,3,1) has a mode of 4.  g4²ÊLÍ3 4€i€‘€€€€‚€€‚ÿ1. Give an efficient and correct algorithm to compute the mode of a set of n numbers.2. Suppose we know that there is an (unknown) element that occurs n/2+1 times in the set. Give a worst-case linear-time algorithm to find the mode. For partial credit, your algorithm may run in expected linear time.š%åË ƒ ԀY€Ȁ†"€Ȇ"€ɀ€€€†"€†"€Ȇ"€ɀ€†"€‡‚†"€ʀ€€€‚ÿ5. Given two sets and (each of size n), and a number x, describe an algorithm for finding whether there exists a pair of elements, one from and one from , that add up to x. (For partial credit, give a algorithm for this problem.)6. For each of the following problems, give an algorithm that finds the desired numbers within the given amount of time. To keep your answers brief, feel free to use algorithms from the book as subroutines. For the example, , 19-3 maximizes the difference, while 8-6 minimizes the difference.LÍ .ðLÍ_ Œ€c€Ȁ€€€€€€†"€ˀ€€€€€€€€€‚ÿ(a) Let S be an unsorted array of n integers. Give an algorithm that finds the pair that maximizes |x-y|. Your algorithm must run in O(n) worst-case time.« Y €€Y€Ȁ€€€€€€†"€ˀ€€€€€€€‚ÿ(b) Let S be a sorted array of n integers. Give an algorithm that finds the pair that maximizes |x-y|. Your algorithm must run in O(1) worst-case time.±5e ˜€i€Ȁ€€€€€€†"€ˀ€€€€€†"€̆"€‚ÿ(c) Let S be an unsorted array of n integers. Give an algorithm that finds the pair that minimizes |x-y|, for . Your algorithm must run in worst-case time.*²_x Ÿ€k€Ȁ€€€€€€†"€ˀ€€€€€†"€̀€€€‚†"€W€€‚ÿ(d) Let S be a sorted array of n integers. Give an algorithm that finds the pair that minimizes |x-y|, for . Your algorithm must run in O(n) worst-case time.7. (*) Describe how to modify any balanced tree data structure such that search, insert, delete, minimum, and maximum still take time each, but successor and predecessor now take O(1) time each. Which operations have to be modified to support this?Îr5-\ †€í€Ȁç˜K€‰€†"€͆"€΂†"€Á†"€Á€€‚ÿ8. (*) In one of my research papers [Ski88], I give a comparison-based sorting algorithm that runs in . Given the existence of an lower bound for sorting, how can this be possible?  9. (*) Suppose you have access to a balanced dictionary data structure, which supports each of the operations search, insert, delete, minimum, maximum, successor, and predecessor in time. Explain how to modify the insert and delete operations so they still take but now minimum and maximum take O(1) time. (Hint: think in terms of using the abstract dictionary operations, instead of mucking about with pointers and the like.)Xù_… _ Œ€õ€Ȁ€€€€€€€€†"€΂€€€€€€€€‚ÿ10. (*) Mr. B. C. Dull claims to have developed a new data structure for priority queues that supports the operations Insert, Maximum, and Extract-Max, all in O(1) worst-case time. Prove that he is mistaken. (Hint: the argument does not involve a lot of gory details-just think about what this would imply about the lower bound for sorting.)11. Use the partitioning step of Quicksort to give an algorithm that finds the median element of an array of n integers in expected O(n) time.R-× 4 6€?€Ȁ€€†"€Á‚ÿ12. (*) You are given the task of reading in n numbers and then printing them out in sorted order. Suppose you have access to a balanced dictionary data structure, which supports each of the operations search, insert, delete, minimum, maximum, successor, and predecessor in time.«… Ø V z€]€‘€€€€†"€‚€€†"€‚€€†"€‚ÿu Explain how you can use this dictionary to sort in time using only the following abstract operations: minimum, successor, insert, search.u Explain how you can use this dictionary to sort in time using only the following abstract operations: minimum, insert, delete, search.u Explain how you can use this dictionary to sort in time using only the following abstract operations: insert and in-order traversal.2ú×  8 >€ù€Ȁ†"€†"€‡‚ÿ13. The running time for Quicksort depends upon both the data being sorted and the partition rule used to select the pivot. Although Quicksort is on average, certain partition rules cause Quicksort to take time if the array is already sorted.é¢Ø  @G \€K€Ȁ€€†"€φ"€І"€‡‚ÿ(a) Suppose we always pick the pivot element to be the key from the last position of the subarray. On a sorted array, does Quicksort now take , , or ?  @.Ãë€ ÷@G \€O€Ȁ€€†"€φ"€І"€‡‚ÿ(b) Suppose we always pick the pivot element to be the key from the middle position of the subarray. On a sorted array, does Quicksort now take , , or ?Z  @QBM h€!€Ȁ€€€€†"€φ"€І"€‡‚ÿ(c) Suppose we always pick the pivot element to be the key of the median element of the first three keys of the subarray. (The median of three keys is the middle value, so the median of 5, 3, 8 is five.) On a sorted array, does Quicksort now take , , or ?ƒ°÷@ÔFÓ tk€Ȁ€€†"€φ"€І"€‡‚€€€€€€†"€р€€€€€‚€€€€†"€Ҁ€€€€€€€€€€€€€€€€€€€€€€€‚ÿ(d) Suppose we always pick the pivot element to be the key of the median element of the first, last, and middle elements of the subarray. On a sorted array, does Quicksort now take , , or ?14. (*) Given a set S of n integers and an integer T, give an algorithm to test whether k of the integers in S sum up to T.15. (**) Design a data structure that allows one to search, insert, and delete an integer X in O(1) time (i.e. constant time, independent of the total number of integers stored). Assume that and that there are m+n units of space available for the symbol table, where m is the maximum number of integers that can be in the table at any one time. (Hint: use two arrays A[1..n] and B[1..m].) You are not allowed to initialize either A or B, as that would take O(m) or O(n) operations. This means the arrays are full of random garbage to begin with, so you must be very careful.4ÜQBIX ~€»€Ȁ€€€€€€€€€€€€€€†"€‚ÿ16. (*) Let P be a simple, but not necessarily convex, polygon and q an arbitrary point not necessarily in P. Design an efficient algorithm to find a line segment originating from q that intersects the maximum number of edges of P. In other words, if standing at point q, in what direction should you aim a gun so the bullet will go through the largest number of walls. A bullet through a vertex of P gets credit for only one wall. An algorithm is possible.>ÔFFJ: B€ €Ȁ€€€€†"€‚ÿ17. (**) The onion of a set of n points is the series of convex polygons that result from finding the convex hull, striping it from the point set, and repeating until no more points are left. Give an algorithm for determining the onion of a point set.,IrJ' € €€€‚ÿ)FJ›J& €€€‚ÿÎÍrJiL Ё«€€‚çln_ɀ†"€ ‰€çgxÊ«€†"€ ‰€ç£8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æé°Û/€†"€‰€æê°Û/€†"€‰€æë°Û/€†"€‰€€çln_ɀ‰€€çgxÊ«€‰€€ç£8ۀ‰€€‚‚ÿ Next: Implementation Challenges Up: Data Structures and Sorting Previous: War Story: String 'em AlgorithmsMon Jun 2 23:33:50 EDT 1997)›J’L& €€€‚ÿµfiLGMO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1’LxM1UÿÿÿÿÿÿÿÿeÿÿÿÿxMœM$GMœM" €€€ ÿDxMàM1l ÿÿÿÿÿÿÿÿfàMNn‰Dynamic Programming:œMN# €.€€‚ÿDynamic Programming "àMH ؁K€€çŠ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æì°Û/€†"€‰€æí°Û/€†"€‰€æ±Û/€†"€‰€€çŠ8ۀ‰€€çÿ7ۀ‰€€çÿ7ۀ‰€‚€‚€‚ÿ Next: Fibonacci numbers Up: Breaking Problems Down Previous: Breaking Problems DownDynamic ProgrammingAfter you understand it, dynamic programming is proNHœMbably the easiest algorithm design technique to apply in practice. In fact, I find that dynamic programming algorithms are usually easier to reinvent than to try to look up in a book. Until you understand it, however, dynamic programming seems like magic. You have to figure out the trick before you can use it.   a(N©„9 @€Q€€€€€€€‚€‚ÿIn algorithms for problems such as sorting, correctness tends to be easier to verify than efficiency. This is not the case for optimization problems, where we seek to find a solution that maximizes or minimizes some function. In designing algorithms for an optimization problem, we must prove that our algorithm always gives the best possible solution. Greedy algorithms, which make the best local decision at each step, occasionally happen to produce a global optimum for certain problems. These are typically efficient. However, you need a proof to show that you always end up with the best answer. Exhaustive search algorithms, which try all possibilities and select the best, by definition must always produce the optimum result, but usually at a prohibitive cost in terms of time complexity.à²H‰†. *€e€€€€€‚ÿDynamic programming combines the best of both worlds. The technique systematically considers all possible decisions and always selects the one that proves to be the best. By storing the consequences of all possible decisions to date and using this information in a systematic way, the total amount of work is minimized. Dynamic programming is best learned by carefully studying a number of examples until things start to click.)©„²†& €€€‚ÿ&‰†؆# €€€‚ÿ¥²†î‡q °€K€Ȁçù㥀‰€‚€çZr€‰€‚€ç»!c€‰€‚€ç'€‰€‚€ç}.!€‰€‚ÿu Fibonacci numbers u The Partition Problem u Approximate String Matching u Longest Increasing Sequence u Minimum Weight Triangulation&؆ˆ# €€€‚ÿ)î‡=ˆ& €€€‚ÿS+ˆˆ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)=ˆ¹ˆ& €€€‚ÿµfˆn‰O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¹ˆŸ‰1UÿÿÿÿÿÿÿÿgÿÿÿÿŸ‰É$n‰É" €€€ ÿBŸ‰Š1ãÿÿÿÿÿÿÿÿhŠ=ŠöÉFibonacci numbers8É=Š# €*€€‚ÿFibonacci numbers PEŠŽ  䁛€€ç§8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ±Û/€†"€‰€æ±Û/€†"€‰€æ±Û/€†"€‰€€ç§8ۀ‰€€ç¥8ۀ‰€€ç¥8ۀ‰€‚€ ‚€€€‚ÿ Next: The Partition Problem Up: Dynamic Programming Previous: Dynamic ProgrammingFibonacci numbersThe tradeoff between space and time exploited in dynamic programming is best illustrated in evaluating recurrence relations, such as the Fibonacci numbers. The Fibonacci numbers were originally defined by the Italian mathematician Fibonacci in the thirteenth century to model the growth of rabbit populations. Rabbits breed, well, like rabbits. Fibonacci surmised that the number of pairs of rabbits born in a given year is equal to the number of pairs of rabbits born in each of the two previous years, if you start with one pair of rabbits in the first year. To count the number of rabbits born in the nth year, he defined the following recurrence relation:    Á\=ŠZÁe ˜€Å€€†"€ӂ†"€Ԇ"€Ն"€ֆ"€׆"€؂€€‚ÿwith basis cases and . Thus , , and the series continues . As it turns out, Fibonacci's formula didn't do a very good job of counting rabbits, but it does have a host of other applications and interesting properties.Since they are defined by a recursiveŽZÁÉ formula, it is easy to write a recursive program to compute the nth Fibonacci number. Most students have to do this in one of their first programming courses. Indeed, I have particularly fond memories of pulling my hair out writing such a program in 8080 assembly language. In pseudocode, the recursive algorithm looks like this:à2Ž:Ä® *o€€‚€€‚€€‚€€‚€€€€‚€†"€ـ€‚€€†"€چ"€Ûçô㥀†"€‰€€€€€†"€܂ÿ Fibonacci[n] if (n=0) then return(0) else if (n=1) then return(1) else return(Fibonacci[n-1]+Fibonacci[n-2])   Figure: The computation tree for computing Fibonacci numbers recursively  How much time does this algorithm take to compute Fibonacci[n]? Since , this means that . Since our recursion tree, illustrated in Figure , has only 0 and 1 as leaves, summing up to such a large number means we must have at least leaves or procedure calls! This humble little program takes exponential time to run!ºŒZÁôÄ. *€€€†"€݂ÿIn fact, we can do much better. We can calculate in linear time by storing all values. We trade space for time in the algorithm below:šF:ÄœÅb ”€’€€‚€€‚†"€ԁ‚†"€Ձ‚€€€€†"€ށ‚ÿ Fibonacci[n] For i=1 to n, “>ôÄ/ÇU x€ƒ€€†"€߆"€à†"€á€€€€€€€‚ÿBecause we evaluate the Fibonacci numbers from smallest to biggest and store all the results, we know that we have and ready whenever we need to compute . Thus each of the n values is computed as the simple sum of two integers in total O(n) time, which is quite an improvement over exponential time.  )œÅXÇ& €€€‚ÿÀ¿/ÇÉ Ё€€‚ç§8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ±Û/€†"€‰€æ±Û/€†"€‰€æ±Û/€†"€‰€€ç§8ۀ‰€€ç¥8ۀ‰€€ç¥8ۀ‰€€‚‚ÿ Next: The Partition Problem Up: Dynamic Programming Previous: Dynamic Programming AlgorithmsMon Jun 2 23:33:50 EDT 1997)XÇAÉ& €€€‚ÿµfÉöÉO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1AÉ'Ê1Uÿÿÿÿÿÿÿÿiÿÿÿÿ'ÊKÊ$öÉKÊ" €€€ ÿF'Ê‘Ê1¥'ÿÿÿÿÿÿÿÿj‘ÊÍÊqThe Partition Problem<KÊÍÊ# €2€€‚ÿThe Partition Problem MF‘ÊÎ ܁€€çš8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€çŠ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ±Û/€†"€‰€æ±Û/€†"€‰€æ±Û/€†"€‰€€çš8ۀ‰€€ç¥8ۀ‰€€çŠ8ۀ‰€‚€ ‚€‚‚ÿ Next: Approximate String Matching Up: Dynamic Programming Previous: Fibonacci numbersThe Partition Problem Suppose that three workers are given the task of scanning through a shelf of books in search of a given piece of information. To get the job done fairly and efficiently, the books are to be partitioned among the three workers. To avoid the need to rearrange the books or separate them into piles, it would be simplest to divide the shelf into three regions and assign each region to one worker.     WÍÊ}A P€1€€‚†"€₁‚‚†"€め‚ÿBut what is the fairest way to divide the shelf up? If each book is the same length, say 100 pages, the job is pretty easy. Just partition the books into equal-sized regions,so that everyone has 300 pages to deal with.But what if the books are not the same length? Suppose we used the same partition when the book sizes looked like this:I, for one, would volunteer to take the first section, with only 600 pÎ}KÊages to scan, instead of the last one, with 2,400 pages. The fairest possible partition for this shelf would beº<Î7~ ʀ€€†"€䂁‚€‚€€€†"€å€€€‚€€€€€€€çÇIÝ*€†"€‰€‚ÿwhere the largest job is only 1,700 pages and the smallest job 1,300.In general, we have the following problem: Input: A given arrangement S of non-negative numbers and an integer k. Output: Partition S into k ranges, so as to minimize the maximum sum over all the ranges. This so-called linear partition problem arises often in parallel processing, since we seek to balance the work done across processors so as to minimize the total elapsed run time. Indeed, the war story of Section revolves around a botched solution to this problem.†_}œ' €Ÿ€€€‚ÿStop for a few minutes and try to find an algorithm to solve the linear partition problem.)7æ& €€€‚ÿÉKœ¯~ ʀŸ€€‚€†"€悁€€€€€€€€€€†"€ç†"€è€€€€†"€é€€‚ÿThe beginning algorist might suggest a heuristic as the most natural approach to solve the partition problem. Perhaps they would compute the average size of a partition, , and then try to insert the dividers so as to come close to this average. However, such heuristic methods are doomed to fail on certain inputs, because they do not systematically evaluate all possibilities.Instead, consider a recursive, exhaustive search approach to solving this problem. Notice that the kth partition starts right after we placed the (k-1)st divider. Where can we place this last divider? Between the ith and (i+1)st elements for some i, where . What is the cost of this? The total cost will be the larger of two quantities, (1) the cost of the last partition and (2) the cost of the largest partition cost formed to the left of i. What is the size of this left partition? To minimize our total, we would want to use the k-2 remaining dividers to partition the elements as equally as possible. This is a smaller instance of the same problem, and hence can be solved recursively! þ“æ­ k €€/€€€€€€€€†"€å€€‚†"€ꂁ‚†"€낁†"€삁‚ÿTherefore, let us define M[n,k] to be the minimum possible cost over all partitionings of into k ranges, where the cost of a partition is the largest sum of elements in one of its parts. Thus defined, this function can be evaluated:with the natural basis cases ofBy definition, this recurrence must return the size of the optimal partition. But how long does it take? If we evaluate the recurrence without storing partial results, we will be doomed to spend exponential time, repeatedly recalculating the same quantities. However, by storing the computed values and looking them up as needed, we can save an enormous amount of time.Pì¯ý d –€ß€€†"€í€€€€€€€€€€€€†"€P†"€î‚ÿHow long does it take to compute this when we store the partial results? Well, how many results are computed? A total of cells exist in the table. How much time does it take to compute the result M[n',k']? Well, calculating this quantity involves finding the minimum of n' quantities, each of which is the maximum of the table lookup and a sum of at most n' elements. If filling each of k n boxes takes at most time per box, the total recurrence can be computed in time.O­ ˜@@ N€¡€€€€†"€ï€€€€‚ÿTo complete the implementation, we must specify the boundary conditions of the recurrence relation and an order to evaluate it in. These boundary conditions always settle the smallest possible values for each of the arguments of the recurrence. For this problem, the smallest reasonable value of the first argument is n=1, meaning that the first partition consists of a single element. We can't create a first partition smaller than regardless oý ˜@KÊf how many dividers are used. The smallest reasonable value of the second argument is k=1, implying that we do not partition S at all.à»ý xA% €w€€‚ÿThe evaluation order computes the smaller values before the bigger values, so that each evaluation has what it needs waiting for it. Full details are provided in the pseudocode below:;˜@ŠC× |€€‚€€‚†"€ð‚€€‚€€€€†"€ñ‚‚€€€€€€€€€€€€‚€€€€†"€ò‚‚€€€€‚€€€€‚†"€󁁂ÿ Partition[S,k] (* compute prefix sums: *) p[0] = 0 for i=1 to n do (* initialize boundary conditions *) for i=1 to n do M[i,1] = p[i] for i=1 to k do (* evaluate main recurrence *) for i=2 to n do for j = 2 to k do xAžG ð€€€€€€‚†"€ô‚€€€€€€‚€€€€€€‚€€€€€€€€‚€†"€õ€€€€€€†"€ö‚†"€÷€€€€€€†"€ø‚†"€€€€€€€†"€ù†"€ú†"€û‚ÿ for x = 1 to i-1 do if (M[i,j] > ) then M[i,j] = D[i,j] = x   Figure: Dynamic programming matrices M and D in partitioning    Figure: Dynamic programming matrices M and D in partitioning  The implementation above in fact runs faster than advertised. Our original analysis assumed that it took time to update each cell of the matrix. This is because we selected the best of up to n possible points to place the divider, each of which requires the sum of up to n possible terms. In fact, it is easy to avoid the need to compute these sums by storing the set of n prefix sums , since . This enables us to evaluate the recurrence in linear time per cell, yielding an algorithm.žŠC·Ia €u€€çVr€†"€‰€çXr€†"€‰€€€€€€€‚ÿBy studying the recurrence relation and the dynamic programming matrices of Figures and , you should be able to convince yourself that the final value of M(n,k) will be the cost of the largest range in the optimal partition. However, what good is that? For most applications, what we need is the actual partition that does the job. Without it, all we are left with is a coupon for a great price on an out-of-stock item.܍žG“KO l€€€€€€€€€€€€€€€€€‚ÿThe second matrix, D, is used to reconstruct the optimal partition. Whenever we update the value of M[i,j], we record which divider position was required to achieve that value. To reconstruct the path used to get to the optimal solution, we work backward from D[n,k] and add a divider at each specified position. This backwards walking is best achieved by a recursive subroutine:|·IšN‹ ä€ý€€‚€€‚€€†"€偁‚‚€€€€€€€€€€€€‚€€†"€ü‚€€‚ÿ ReconstructPartition(S,D,n,k) If (k = 1) then print the first partition else ReconstructPartition(S,D,D[n,k],k-1) Print the kth partition { } It is important to catch the distinction between storing the value of a cell and what decision/move it took to get there. The latter is not used in the computation but is presumably the real thing that you are interested in. For most of the examples in this chapter, we will not worry about reconstructing the answer. However, study this example closely to ensure that you know how to obtain the winning configuration when you need it.)“KÃN& €€€‚ÿÄÚN“€ Ё—€€‚çš8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€çŠ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æD±Û/€†"€‰€æ;±Û/€†"€‰€æ<±Û/€†"€‰€€çš8ۀ‰€€ç¥8ۀ‰€€çŠ8ۀ‰€€‚‚ÿ Next: ÃN“€KÊApproximate String Matching Up: Dynamic Programming Previous: Fibonacci numbers AlgorithmsMon Jun 2 23:33:50 EDT 1997)ÃNŒ€& €€€‚ÿµf“€qO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Œ€¢1Uÿÿÿÿÿÿÿÿkÿÿÿÿ¢Ɓ$qƁ" €€€ ÿL¢‚1o'ÿÿÿÿÿÿÿÿl‚T‚£Approximate String MatchingBƁT‚# €>€€‚ÿApproximate String Matching ÚÓ‚.… ܁·€€ç©8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€ç§8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ=±Û/€†"€‰€æ>±Û/€†"€‰€æ?±Û/€†"€‰€€ç©8ۀ‰€€ç¥8ۀ‰€€ç§8ۀ‰€‚€ ‚€‚‚ÿ Next: Longest Increasing Sequence Up: Dynamic Programming Previous: The Partition ProblemApproximate String Matching An important task in text processing is string matching - finding all the occurrences of a word in the text. Unfortunately, many words in documents are mispelled (sic). How can we search for the string closest to a given pattern in order to account for spelling errors?     IúT‚w†O l€õ€€€€€€€€€€€€€€€€‚ÿTo be more precise, let P be a pattern string and T a text string over the same alphabet. The edit distance between P and T is the smallest number of changes sufficient to transform a substring of T into P, where the changes may be:Tç.…ˇm š€Õ€Ȁ€€†"€ý‚€€€€€€†"€ý‚€€€€€€†"€ý‚ÿ1. Substitution - two corresponding characters may differ: KAT CAT.2. Insertion - we may add a character to T that is in P: CT CAT.3. Deletion - we may delete from T a character that is not in P: CAAT CAT.Iw†‹H ^€€€€€€€‚çÜj€†"€‰€‚ÿFor example, P=abcdefghijkl can be matched to T=bcdeffghixkl using exactly three changes, one of each of the above types.Approximate string matching arises in many applications, as discussed in Section . It seems like a difficult problem, because we have to decide where to delete and insert characters in pattern and text. But let us think about the problem in reverse. What information would we like to have in order to make the final decision; i.e. what should happen with the last character in each string? The last characters may be either be matched, if they are identical, or otherwise substituted one for the other. The only other options are inserting or deleting a character to take care of the last character of either the pattern or the text.2ΡFŒd –€Ÿ€€€€€€€€†"€þ€€€€€€€€€€€€‚ÿMore precisely, let D[i,j] be the minimum number of differences between and the segment of T ending at j. D[i, j] is the minimum of the three possible ways to extend smaller strings:ˆ‹Ύ… ؀ €Ȁ†"€ÿ€€€€€€€€€€€€€€€€‚€€€€€€‚€€€€€€‚ÿ1. If , then D[i-1, j-1], else D[i-1, j-1]+1. This means we either match or substitute the ith and jth characters, depending upon whether they do or do not match.2. D[i-1, j]+1. This means that there is an extra character in the pattern to account for, so we do not advance the text pointer and pay the cost of an insertion.3. D[i, j-1]+1. This means that there is an extra character in the text to remove, so we do not advance the pattern pointer and pay the cost of a deletion.–FŒpÃy À€;€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿThe alert reader will notice that we have not specified the boundary conditions of this recurrence relation. It is critical to get the initialization right if our program is to returnΎpÃƁ the correct edit distance. The value of D[0,i] will correspond to the cost of matching the first i characters of the text with none of the pattern. What this value should be depends upon what you want to compute. If you seek to match the entire pattern against the entire text, this means that we must delete the first i characters of the text, so D[0,i] = i to pay the cost of the deletions. But what if we want to find where the pattern occurs in a long text? It should not cost more if the matched pattern starts far into the text than if it is near the front. Therefore, the starting cost should be equal for all positions. In this case, D[0,i] = 0, since we pay no cost for deleting the first i characters of the text. In both cases, D[i,0] = i, since we cannot excuse deleting the first i characters of the pattern without penalty.ÃeΎ3Å^ Š€Ï€€†"€€€‚†"€€€€€€€€€€€‚ÿ  Figure: Example dynamic programming matrix for edit distance computation, with the optimal alignment path highlighted in bold  Once you accept the recurrence, it is straightforward to turn it into a dynamic programming algorithm that creates an matrix D, where n = |P| and m = |T|. Here it is, initialized for full pattern matching:ÕpÃÇÑ p €€‚€€‚‚€€€€€€€€€€‚€€€€€€€€€€‚‚€€€€‚€€€€‚†"€‚€€€€€€€€€€€€‚ÿ EditDistance(P,T) (*initialization*) For i = 0 to n do D[i,0] = i For i = 0 to m do D[0,i] = i (*recurrence*) For i = 1 to n do For j = 1 to m do D[i-1,j]+1, D[i,j-1]+1 ) ÈM3ÅÐÊ{ ĀŸ€€€€€€€€€€€€‚€€€€€€ç·!c€†"€‰€†"€€€‚ÿHow much time does this take? To fill in cell D[i,j], we need only compare two characters and look at three other cells. Since it requires only constant time to update each cell, the total time is O(mn).The value to return as the answer to our pattern matching problem depends on what we are interested in. If we only needed the cost of comparing all of the pattern against all of the text, such as in comparing the spelling of two words, all we would need is the cost of D[n,m], as shown in Figure . But what if we need to identify the best matching substring in the text? Assuming that the initialization was performed correctly for such substring matching, we seek the cheapest matching of the full pattern ending anywhere in the text. This means the cost equals , i.e. the smallest cost on the last row of D.   ÓÇÚÎ7 <€§€€€€€€€€‚ÿOf course, this only gives the cost of the optimal matching, while we are often interested in reconstructing the actual alignment - which characters got matched, substituted, and deleted. These can be reconstructed from the pattern/text and table without an auxiliary storage, once we have identified the cell with the lowest cost. From this cell, we want to walk upwards and backwards through the matrix. Given the costs of its three neighbors and the corresponding characters, we can reconstruct which choice was made to get to the goal cell. The direction of each backwards step (to the left, up, or diagonal to the upper left) identifies whether it was an insertion, deletion, or match/substitution. Ties can be broken arbitrarily, since either way costs the same. We keep walking backwards until we hit the end of the matrix, specifying the starting point. This backwards-walking phase takes O(n+m) time, since we traverse only the cells involved in the alignment.)ìÐÊ= H€Ù€€€€€€€€€€‚ÿThe alert reader will notice that it is unnecessary to keep all O(mn) cells to compute the cost of an alignment. If we evaluate the recurrence by filling in the columns of the matrix from left to right, we will never need more ÚÎƁthan two columns of cells to store what is necessary for the computation. Thus O(m) space is sufficient to evaluate the recurrence without changing the time complexity at all. Unfortunately, without the full matrix we cannot reconstruct the alignment.  ÅXÚÎÔm š€³€€€€€€€€€€€€€€€€€€çÜj€†"€‰€€‚ÿSaving space in dynamic programming is very important. Since memory on any computer is limited, O(nm) space proves more of a bottleneck than O(nm) time. Fortunately, there is a clever divide-and-conquer algorithm that computes the actual alignment in O(nm) time and O(m) space. This algorithm is discussed in Section .)ý& €€€‚ÿÈÇÔÅ ЁŸ€€‚ç©8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€ç§8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ@±Û/€†"€‰€æA±Û/€†"€‰€æB±Û/€†"€‰€€ç©8ۀ‰€€ç¥8ۀ‰€€ç§8ۀ‰€€‚‚ÿ Next: Longest Increasing Sequence Up: Dynamic Programming Previous: The Partition Problem AlgorithmsMon Jun 2 23:33:50 EDT 1997)ýî& €€€‚ÿµfÅ£O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1îÔ1UÿÿÿÿÿÿÿÿmÿÿÿÿÔø$£ø" €€€ ÿLÔD1oÿÿÿÿÿÿÿÿnD†“OLongest Increasing SequenceBø†# €>€€‚ÿLongest Increasing Sequence +$D± ܁Y€€çª8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€çš8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æC±Û/€†"€‰€æo±Û/€†"€‰€æf±Û/€†"€‰€€çª8ۀ‰€€ç¥8ۀ‰€€çš8ۀ‰€‚€ ‚€‚‚ÿ Next: Minimum Weight Triangulation Up: Dynamic Programming Previous: Approximate String MatchingLongest Increasing Sequence Hopefully, a pattern is emerging. Every dynamic programming solution has three components:  d<† ( €y€Ȁ‚‚‚ÿ1. Formulate the answer as a recurrence relation or recursive algorithm.2. Show that the number of different values of your recurrence is bounded by a (hopefully small) polynomial.3. Specify an order of evaluation for the recurrence so you always have the partial results you need available when you need them.б+ F Z€£€€€€ç`€7€†"€‰€€€‚ÿTo see how this is done, let's see how we would develop an algorithm to find the longest monotonically increasing sequence in a sequence of n numbers. This problem arises in pattern matching on permutations, as described in Section . We distinguish an increasing sequence from a run, in that the selected elements need not be neighbors of each other. The selected elements must be in sorted order from left to right. For example, consider the sequence*ã UG \€Ë€€†"€‚‚€€€€†"€‚ÿThe longest increasing subsequence of has length 3 and is either (2,3,4) or (2,3,6). The longest increasing run is of length 2, either (2,8) or (1,6).Finding the longest increasing run in a numerical sequence is straightforward, indeed you should be able to devise a linear-time algorithm fairly easily. However, finding the longest increasing subsequence is considerably trickier. How can we identify which scattered elements to skip? To apply dynamic programming, we need to construct a recurrence computing the length of the longest sequence. To find the right recurrence, ask what information about the first n-1 elements of S, coupled with the last element , would enable you to find the answer for the entire sequence?Œ+ yA\ †€€Ȁ€€†"€€€†"€†"€†"€€€‚ÿu The length of the longest increasing sequence in seems a useful thing tUyAøo know. In fact, this will be the longest increasing sequence in S, unless extends it or another increasing sequence of the same length. Unfortunately, knowing just this length is not enough. Suppose I told you that the longest increasing sequence in was of length 5 and that . Will the length of the final longest increasing subsequence of S be 5 or 6?ÉUyB7 <€•€Ȁ€†"€€€‚ÿu Therefore, we also need the length of the longest sequence that will extend. To be certain we know this, we really need the length of the longest sequence that any possible number can extend.ã¬yA\C7 <€]€€†"€†"€ ‚ÿThis provides the idea around which to build a recurrence. Define to be the length of the longest sequence ending with . Verify that the following table is correct:+yBtDí#ªX PÌ Ì Ì Ì Ì Ì Ì Ì Ì Ì €€ÿ(€€€†"€ ‚ÿ€€‚ÿ€ €‚ÿ€&€‚ÿ€,€‚ÿ€2€‚ÿ€8€‚ÿ€>€‚ÿ€D€‚ÿ€J€‚ÿ €P€‚ÿÿÿsequence 952873164*\CŒEî#¬V PÌ Ì Ì Ì Ì Ì Ì Ì Ì Ì €€€‚ÿ"€€†"€‚ÿ€€‚ÿ€€‚ÿ€$€‚ÿ€*€‚ÿ€0€‚ÿ€6€‚ÿ€<€‚ÿ€B€‚ÿ€H€‚ÿ €N€‚ÿÿÿlength 111222133.2tDºFü#ȁf PÌ Ì Ì Ì Ì Ì Ì Ì Ì Ì €€€‚ÿ"€€†"€&‚ÿ€$€‚ÿ€(€‚ÿ€.€‚ÿ€4€‚ÿ€:€‚ÿ€@€‚ÿ€F€‚ÿ€L€‚ÿ€R€‚ÿ €X€‚ÿ €^€‚ÿÿÿpredecessor ---223-66'ÏŒEáHX ~€§€€€€€€†"€†"€‚†"€ ‚†"€ ‚ÿThe longest increasing sequence containing the nth number will be formed by appending it to the longest increasing sequence to the left of n that ends on a number smaller than . The following recurrence computes :These values define the length of the longest increasing sequence ending at each number. The length of the longest increasing subsequence of the entire permutation is given by , since the winning sequence will have to end somewhere.Ü[ºFœL ЀÉ€€€€†"€†"€ †"€ †"€†"€‚†"€ †"€&†"€ †"€ €‚ÿWhat is the time complexity of this algorithm? Each one of the n values of is computed by comparing against up to values to the left of it, so this analysis gives a total of time. In fact, by using dictionary data structures in a clever way, we can evaluate this recurrence in time. However, the simple recurrence would be easy to program and therefore is a good place to start.What auxiliary information will we need to store in order to reconstruct the actual sequence instead of its length? For each element , we will store the index of the element that appears immediately before in the longest increasing sequence ending at . Since all of these pointers go towards the left, it is a simple matter to start from the last value of the longest sequence and follow the pointers so as to reconstruct the other items in the sequence.)áHæL& €€€‚ÿÏÎœLµN Ё­€€‚çª8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€çš8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æg±Û/€†"€‰€æh±Û/€†"€‰€æi±Û/€†"€‰€€çª8ۀ‰€€ç¥8ۀ‰€€çš8ۀ‰€€‚‚ÿ Next: Minimum Weight Triangulation Up: Dynamic Programming Previous: Approximate String Matching AlgorithmsMon Jun 2 23:33:50 EDT 1997)æLÞN& €€€‚ÿµfµN“OO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÞNÄO1UÿÿÿÿÿÿÿÿoÿÿÿÿÄO €$“O €" €€€ ÿÄO €“OMÄOY€1ÉÿÿÿÿÿÿÿÿpY€œ€¿ÂMinimum Weight TriangulationC €œ€# €@€€‚ÿMinimum Weight Triangulation /Y€˂ üC€€çÖ8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€ç©8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æj±Û/€†"€‰€æk±Û/€†"€‰€æl±Û/€†"€‰€€çÖ8ۀ‰€€ç¥8ۀ‰€€ç©8ۀ‰€‚€ ‚€‚†"€ €€‚ÿ Next: Limitations of Dynamic Programming Up: Dynamic Programming Previous: Longest Increasing SequenceMinimum Weight Triangulation   Figure: Two different triangulations of a given convex seven-gon  8Ôœ€…d –€¯€€€€†"€€€çv.!€†"€‰€çoXœ€€†"€‰€‚ÿA triangulation of a polygon is a set of non-intersecting diagonals that partitions the polygon into triangles. We say that the weight of a triangulation is the sum of the lengths of its diagonals. As shown in Figure , any given polygon may have many different triangulations. For any given polygon, we seek to find its minimum weight triangulation. Triangulation is a fundamental component of most geometric algorithms, as discussed in Section .  kÃ˂n‰š —€€†"€‚†"€€€€€€€€€‚çy.!€†"€‰€€€€€€€†"€†"€†"€†"€†"€‚ÿTo apply dynamic programming, we need a way to carve up the polygon into smaller pieces. A first idea might be to try all possible chords, each of which partitions the polygon into two smaller polygons. Using dynamic programming, this will work to give a polynomial-time algorithm. However, there is a slicker approach.  Figure: Selecting the vertex k to pair with an edge (i,j) of the polygon  Observe that every edge of the input polygon must be involved in exactly one triangle. Turning this edge into a triangle means identifying the third vertex, as shown in Figure . Once we find the correct connecting vertex, the polygon will be partitioned into two smaller pieces, both of which need to be triangulated optimally. Let T[i,j] be the cost of triangulating from vertex to vertex , ignoring the length of the chord from to . The latter clause avoids double counting these internal chords in the following recurrence:z…èŠ\ †€?€€†"€‚€€€€€€€€€€‚€€€€‚ÿThe basis condition applies when i and j are immediate neighbors, as T[i,i+1] = 0.Since the number of vertices in each subrange of the right side of the recurrence is smaller than that on the left side, evaluation can proceed in terms of the gap size from i to j:wn‰_ö º €€‚€€‚€€€€€€€€€€‚€€€€‚€€€€‚€€€€€€‚†"€‚€€€€‚€†"€€€€€€€€€€€€€€€€€†"€Q†"€‚ÿ Minimum-Weight-Triangulation(P) for i=1 to n do T[i, j]=0 for gap=1 to n-1 for i=1 to n-gap do j=i+gap return T[1, n] There are values of T, each of which takes O(j-i) time if we evaluate the sections in order of increasing size. Since j-i = O(n), complete evaluation takes time and space.\+芻1 0€Y€€†"€€‚ÿWhat if there are points in the interior of the polygon? Then dynamic programming does not apply in the same way, because each of the triangulation edges does not necessarily cut the boundary into two distinct pieces as before. Instead of only possible subregions, the number of subregions now grows exponentially. In fact, no efficient algorithm for this problem is known. More surprisingly, there is also no known proof of hardness. Minimum weight triangulation is one of the few well-studied algorithmic problems that remain in this limbo state.)_ À& €€€‚ÿ» À €ÕÔ»áÁ Ё¹€€‚çÖ8ۀ†"€ ‰€ç¥8ۀ†"€ ‰€ç©8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æm±Û/€†"€‰€æn±Û/€†"€‰€暱Û/€†"€‰€€çÖ8ۀ‰€€ç¥8ۀ‰€€ç©8ۀ‰€€‚‚ÿ Next: Limitations of Dynamic Programming Up: Dynamic Programming Previous: Longest Increasing Sequence AlgorithmsMon Jun 2 23:33:50 EDT 1997) À Â& €€€‚ÿµfáÁ¿ÂO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 ÂðÂ1UÿÿÿÿÿÿÿÿqÿÿÿÿðÂÃ$¿ÂÃ" €€€ ÿS"ðÂgÃ1PÿÿÿÿÿÿÿÿrgðÃÀLimitations of Dynamic ProgrammingI&ðÃ# €L€€‚ÿLimitations of Dynamic Programming å°gÕÈ5 8‚q€€çÍ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çª8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€摱Û/€†"€‰€撱Û/€†"€‰€擱Û/€†"€‰€€çÍ8ۀ‰€€çÿ7ۀ‰€€çª8ۀ‰€‚€‚€€€€€€€€€€€€€€€€€‚ÿ Next: War Story: Evolution of Up: Breaking Problems Down Previous: Minimum Weight TriangulationLimitations of Dynamic ProgrammingDynamic programming can be applied to any problem that observes the principle of optimality. Roughly stated, this means that partial solutions can be optimally extended with regard to the state after the partial solution instead of the partial solution itself. For example, to decide whether to extend an approximate string matching by a substitution, insertion, or deletion, we do not need to know exactly which sequence of operations was performed to date. In fact, there may be several different edit sequences that achieve a cost of C on the first p characters of pattern P and t characters of string T. Future decisions will be made based on the consequences of previous decisions, not the actual decisions themselves.  †Y°ÃÍ- (€³€€‚€€‚ÿProblems do not satisfy the principle of optimality if the actual operations matter, as opposed to just the cost of the operations. Consider a form of edit distance where we are not allowed to use combinations of operations in certain particular orders. Properly formulated, however, most combinatorial problems respect the principle of optimality.The biggest limitation on using dynamic programming is the number of partial solutions we must keep track of. For all of the examples we have seen, the partial solutions can be completely described by specifying the stopping places in the input. This is because the combinatorial objects being worked on (strings, numerical sequences, and polygons) all have an implicit order defined upon their elements. This order cannot be scrambled without completely changing the problem. Once the order is fixed, there are relatively few possible stopping places or states, so we get efficient algorithms. If the objects are not firmly ordered, however, we have an exponential number of possible partial solutions and are doomed to need an infeasible amount of memory.ì|•Èp ®€ÿ€€ç¥—K€‰€€€€€€€€€€€†"€€€†"€†"€‚ÿTo illustrate this, consider the following dynamic programming algorithm for the traveling salesman problem, discussed in greater detail in [RND77]. Recall that solving a TSP means finding the order that visits each site exactly once, while minimizing the total distance traveled or cost paid. Let C(i,j) to be the edge cost to travel directly from i to j. Define to be the cost of the optimal tour from i to 1 that goes through each of the cities exactly once, in any order. The cost of the optimal TSP tour is thus defined to be and can be computed recursively by identifying the first edge in this sequÍÃence:Ø`Íëx Ÿ€Í€€†"€‚‚†"€‚‚†"€†"€U€€†"€†"€€€€€€‚ÿusing the basis cases This recurrence, although somewhat complicated to understand, is in fact correct. However, each partial solution is described by a vertex subset . Since there are subsets of n vertices, we require time and space to evaluate this recurrence. Whenever the input objects do not have an inherent left-right order, we are typically doomed to having an exponential-sized state space. Occasionally this is manageable - indeed, is a big improvement over enumerating all O(n!) possible TSP tours. Still, dynamic programming is most effective on well-ordered objects.)& €€€‚ÿÎÍëâ Ё«€€‚çÍ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çª8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€攱Û/€†"€‰€敱Û/€†"€‰€斱Û/€†"€‰€€çÍ8ۀ‰€€çÿ7ۀ‰€€çª8ۀ‰€€‚‚ÿ Next: War Story: Evolution of Up: Breaking Problems Down Previous: Minimum Weight Triangulation AlgorithmsMon Jun 2 23:33:50 EDT 1997) & €€€‚ÿµfâÀO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 ñ1Uÿÿÿÿÿÿÿÿsÿÿÿÿñ$À" €€€ ÿT#ñi1*$ÿÿÿÿÿÿÿÿti³†War Story: Evolution of the LobsterJ'³# €N€€‚ÿWar Story: Evolution of the Lobster ‰„i<  ؁€€çÎ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÖ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€旱Û/€†"€‰€昱Û/€†"€‰€晱Û/€†"€‰€€çÎ8ۀ‰€€çÿ7ۀ‰€€çÖ8ۀ‰€‚€‚€‚ÿ Next: War Story: What's Past Up: Breaking Problems Down Previous: Limitations of Dynamic ProgrammingWar Story: Evolution of the LobsterI'll confess that I'm always surprised to see graduate students stop by my office early in the morning. Something about working through the night makes that time inconvenient for them. But there they were, two future Ph.D.s working in the field of high-performance computer graphics. They studied new techniques for rendering pretty computer images. The picture they painted for me that morning was anything but pretty, however.       4 ³p + $€€€‚‚‚‚ÿ``You see, we want to build a program to morph one image into another,'' they explained.``What do you mean by morph?'' I asked.``For special effects in movies, we want to construct the intermediate stages in transforming one image into another. Suppose we want to turn you into Humphrey Bogart. For this to look realistic, we must construct a bunch of in-between frames that start out looking like you and end up looking like him.''``If you can realistically turn me into Bogart, you have something,'' I agreed.šL< $@\ †€€€‚‚†"€€€‚€€€€猔€†"€‰€‚ÿ``But our problem is that it isn't very realistic.'' They showed me a dismal morph between two images. ``The trouble is that we must find the right correspondence between features in the two images. It looks real bad when we get the correspondence wrong and try to morph a lip into an ear.''``I'll bet. So you want me to give you an algorithm for matching up lips?''  Figure: A successful alignment of two lines of pixels  ``No, even simpler. We just morph each row of the initial image into the identical row of the final image. You can assume that we give you two lines of pixels, and you have to find the best possible match between the dark pixels in a row from object A to the dark pixels in the corresponding row of object B. Like this,'' they said, showing me images of successful matchings like p $@those in Figure .íp :B) €Û€€‚‚‚ÿ``I see,'' I said. ``You want to match big dark regions to big dark regions and small dark regions to small dark regions.''``Yes, but only if the matching doesn't shift them too much to the left or the right. We might prefer to merge or break up regions rather than shift them too far away, since that might mean matching a chin to an eyebrow. What is the best way to do it?''``One last question. Will you ever want to match two intervals to each other in such a way that they cross?''«Z$@åDQ p€¹€€‚ç²!c€†"€‰€çÜj€†"€‰€‚ÿ``No, I guess not. Crossing intervals can't match. It would be like switching your left and right eyes.''I scratched my chin and tried to look puzzled, but I'm just not as good an actor as Bogart. I'd had a hunch about what needed to be done the instant they started talking about lines of pixels. They want to transform one array of pixels into another array, with the minimum amount of changes. That sounded like editing one string of pixels into another string, which is a classic application of dynamic programming. See Sections and for discussions of approximate string matching.É:BôGF Z€•€€€€€€†"€€€€€‚ÿThe fact that the intervals couldn't cross just sealed things up. It meant that whenever a stretch of dark pixels from A was mapped to a stretch from B, the problem would be split into two smaller subproblems, i.e. the pixels to the left of the match and the pixels to the right of the match. The cost of the global match would ultimately be the cost of this match plus those of matching all the pixels to the left and of matching all the pixels to the right. Constructing the optimal match on the left side is a smaller problem and hence simpler. Further, there can be only possible left subproblems, since each is completely described by the pair of one of n top pixels and one of n bottom pixels.>åD2J= H€€€€€€€€€€€‚ÿ``Your algorithm will be based on dynamic programming,'' I pronounced. ``However, there are several possible ways to do things, depending upon whether you want to edit pixels or runs. I would probably convert each row into a list of black pixel runs, with the runs sorted by right endpoint and each run labeled with its starting position and length. You will maintain the cost of the cheapest match between the leftmost i runs and the leftmost j runs for all i and j. The possible edit operations are:KôG±J4 8€˜€€†"€€€‚ÿ  Figure: Morphing a lobster into a head via dynamic programming  ­^2J^MO l€œ€Ȁ€€€€€€€‚€€€‚€€€‚ÿu Full run match: We may match run i on top to run j on the bottom for a cost that is a function of the difference in the lengths of the two runs and their positions.u Merging runs: We may match a string of consecutive runs on top to a run on the bottom. The cost will be a function of the number of runs, their relative positions, and their lengths.u Splitting runs: We may match a big run on top to a string of consecutive runs on the bottom. This is just the converse of the merge. Again, the cost will be a function of the number of runs, their relative positions, and their lengths.J±JšOE X€ €€€€€€€€€€€€‚‚ÿ``For each pair of runs (i,j) and all the cases that apply, we compute the cost of the edit operation and add to the (already computed and stored) edit cost to the left of the start of the edit. The cheapest of these cases is what we will take for the cost of c[i,j].''The pair of graduate students scribbled this down, then frowned. ``So we are going to have a cost measure for matching two runs that is a function of their lengths and positions. How do we decide what the relative costs should be?''ˆ7^M<ƒQ p€q€€€€‚ç³‰K€‰€矔€†"€‰€€‚ÿ``ThašO<ƒt is your business. The dynamic programming serves to optimize the matchings once you know the cost functions. It is up to your aesthetic sense to decide what penalties there should be for line length changes or offsets. My recommendation is that you implement the dynamic programming and try different values for the constants effecting the relative penalties on each of several different images. Then pick the setting that seems to do what you want.''They looked at each other and smiled, then ran back into the lab to implement it. Using dynamic programming to do their alignments, they completed their morphing system, which is described in [HWK94]. They produced the images shown in Figure , morphing a lobster into a man. Unfortunately, they never got around to turning me into Humphrey Bogart.)šOeƒ& €€€‚ÿÓÒ<ƒ8… Ёµ€€‚çÎ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÖ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æþžÛ/€†"€‰€æõžÛ/€†"€‰€æöžÛ/€†"€‰€€çÎ8ۀ‰€€çÿ7ۀ‰€€çÖ8ۀ‰€€‚‚ÿ Next: War Story: What's Past Up: Breaking Problems Down Previous: Limitations of Dynamic Programming AlgorithmsMon Jun 2 23:33:50 EDT 1997)eƒa…& €€€‚ÿµf8…†O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1a…G†1UÿÿÿÿÿÿÿÿuÿÿÿÿG†k†$†k†" €€€ ÿQ G†Œ†1é'ÿÿÿÿÿÿÿÿvŒ†‡U War Story: What's Past is PrologG$k†‡# €H€€‚ÿWar Story: What's Past is Prolog 94Œ†<‰ ؁y€€çÏ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÍ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ÷žÛ/€†"€‰€æøžÛ/€†"€‰€æùžÛ/€†"€‰€€çÏ8ۀ‰€€çÿ7ۀ‰€€çÍ8ۀ‰€‚€‚€‚ÿ Next: War Story: Text Compression Up: Breaking Problems Down Previous: War Story: Evolution ofWar Story: What's Past is Prolog``But our heuristic works very, very well in practice.'' He was simultaneously boasting and crying for help.2Ÿ‡n“ ô€?€€‚€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿUnification is the basic computational mechanism in logic programming languages like Prolog. A Prolog program consists of a set of rules, where each rule has a head and an associated action whenever the rule head matches or unifies with the current computation.      An execution of a Prolog program starts by specifying a goal, say p(a,X,Y), where a is a constant and X and Y are variables. The system then systematically matches the head of the goal with the head of each of the rules that can be unified with the goal. Unification means binding the variables with the constants, if it is possible to match them. For the nonsense program below, p(X,Y,a) unifies with either of the first two rules, since X and Y can be bound to match the extra characters. The goal p(X,X,a) would only match the first rule, since the variable bound to the first and second positions must be the same.™¯<‰ê ¢_€€‚€€€€€€€€€€€€‚€€€€€€€€€€€€€€€€‚€€€€€€€€€€€€€€€€‚€€€€€€€€€€€€€€€€‚ÿ p(a,a,a) := h(a); p(b,a,a) := h(a) * h(b); p(c,b,b) := h(b) + h(c); p(d,b,b) := h(d) + h(b); ÅknØÁZ ‚€Û€€†"€€€‚‚çýùր†"€‰€€€€€‚ÿ  Figure: Two different tries for the given set of rule heads  ``In order to speed up unification, we want to preprocess the set of rule heads so thatØÁk† we can quickly determine which rules match a given goal. We must organize the rules in a trie data structure for fast unification.''Tries are extremely useful data structures in working with strings, as discussed in Section . Every leaf of the trie represents one string. Each node on the path from root to leaf is labeled with exactly one character of the string, with the ith node of the path corresponding to the ith character of the string.  Ð©šÄ' €S€€‚‚ÿ``I agree. A trie is a natural way to represent your rule heads. Building a trie on a set of strings of characters is straightforward - just insert the strings one after another starting from the root. So what is your problem?'' I asked.``The efficiency of our unification algorithm depends very much on minimizing the number of edges in the trie. Since we know all the rules in advance, we have the freedom to reorder the character positions in the rules. Instead of the root node always representing the first argument in the rule, we can choose to have it represent the third argument. We would like to use this freedom to build a minimum-size trie for the set of rules.''HòØÁðÇV z€ç€€ç#~€†"€‰€‚‚€€€€€€€€‚ÿHe showed me the example in Figure . A trie constructed according to the original string position order (1, 2, 3) uses a total of 12 edges. However, by permuting the character order to (2, 3, 1), we can obtain a trie with only 8 edges.``Why does the speed of unification depend on minimizing the number of edges in the trie?''``An open goal is one with all distinct variables in its arguments, like p(X,Y,Z). Open goals match everything. In unifying an open goal against a set of clause heads, each symbol in all the clause heads will be bound to some variable. By doing a depth-first traversal of this minimum-edge trie, we minimize the number of operations in unifying the goal with all of the clause heads,'' he explained quickly."÷šÄÊ+ $€ï€€‚‚‚‚ÿ``Interesting...'' I started to reply before he cut me off again.``One other constraint. For most applications we must keep the leaves of the trie ordered, so that the leaves of the underlying tree go left-to-right in the same order as the rules appear on the page.''``But why do you have to keep the leaves of the trie in the given order?'' I asked.``The order of rules in Prolog programs is very, very important. If you change the order of the rules, the program returns different results.''ϝðÇáÎ2 2€= €€‚‚†"€‚ÿThen came my mission.``We have a greedy heuristic for building good but not optimal tries, based on picking as the root the character position that minimizes the degree of the root. In other words, it picks the character position that has the smallest number of distinct characters in it. This heuristic works very, very well in practice. But we need you to prove that finding the best trie is NP-complete so our paper is, well, complete.''  I agreed to think about proving the hardness of the problem and chased him from my office. The problem did seem to involve some non-trivial combinatorial optimization to build the minimal tree, but I couldn't see how to factor the left-to-right order of the rules into a hardness proof. In fact, I couldn't think of any NP-complete problem that had such a left-right ordering constraint. After all, if a given set of rules contained a character position in common to all the rules, that character position must be probed first in any minimum-size tree. Since the rules were ordered, each node in the subtree must represent the root of a run of consecutive rules, so there were only possible nodes to choose from for this tree....1ŽÊ} Ȁu€€‚‚€€†"€ €€€€†"€!†"€"†"€#†"€$†"€€€€€‚ÿBingo! That settled it.The next day I went back to the professor and told him. ``Look, I cannot prove that your problem is NP-complete. But how would you feeláÎk† about an efficient dynamic programming algorithm to find the best trie!'' It was a pleasure watching his frown change to a smile as the realization took hold. An efficient algorithm to compute what he needed was infinitely better than a proof saying you couldn't do it!My recurrence looked something like this. Suppose that we are given n ordered rule heads , each with m arguments. Probing at the pth position, , partitioned the rule heads into runs , where each rule in a given run had the same character value of . The rules in each run must be consecutive, so there are only possible runs to worry about. The cost of probing at position p is the cost of finishing the trees formed by each of the created runs, plus one edge per tree to link it to probe p:ÖáÎ$0 .€¯€€†"€%‚‚ÿA graduate student immediately set to work implementing dynamic programming to compare with their heuristic. On many programs the optimal and greedy algorithms constructed the exact same trie. However, for some examples, dynamic programming gave a 20% performance improvement over greedy, i.e. 20% better than very, very well in practice. The run time spent in doing the dynamic programming was sometimes a bit larger than with greedy, but in compiler optimization you are always happy to trade off a little extra compilation time for better execution time performance of your program. Is a 20% improvement worth it? That depends upon the situation. For example, how useful would you find a 20% increase in your salary?]I `€+€€€€ç#‚K€€†"€€‰€€‚ÿThe fact that the rules had to remain ordered was the crucial property that we exploited in the dynamic programming solution. Indeed, without it, I was able to prove that the problem was NP-complete, something we put in the paper [DRR 95] to make it complete.  )$ª& €€€‚ÿÍ́w  Ё©€€‚çÏ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÍ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æúžÛ/€†"€‰€æûžÛ/€†"€‰€æüžÛ/€†"€‰€€çÏ8ۀ‰€€çÿ7ۀ‰€€çÍ8ۀ‰€€‚‚ÿ Next: War Story: Text Compression Up: Breaking Problems Down Previous: War Story: Evolution of AlgorithmsMon Jun 2 23:33:50 EDT 1997)ª  & €€€‚ÿµfw U O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1  † 1Uÿÿÿÿÿÿÿÿwÿÿÿÿ† ª $U ª " €€€ ÿZ)†  1N$ÿÿÿÿÿÿÿÿx T TŒWar Story: Text Compression for Bar CodesP-ª T # €Z€€‚ÿWar Story: Text Compression for Bar Codes ¶«    äg€€çÐ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÎ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æýžÛ/€†"€‰€æ{·Û/€†"€‰€ær·Û/€†"€‰€€çÐ8ۀ‰€€çÿ7ۀ‰€€çÎ8ۀ‰€‚€‚€€€‚ÿ Next: Divide and Conquer Up: Breaking Problems Down Previous: War Story: What's PastWar Story: Text Compression for Bar CodesYnjiun waved his laser wand over the torn and crumpled fragments of a bar code label. The system hesitated for a few seconds, then responded with a pleasant blip sound. He smiled at me in triumph. ``Virtually indestructible.''УT æA- (€G€€‚€€‚ÿI was visiting the research laboratories of Symbol Technologies, of Bohemia NY, the world's leading manufacturer of bar code scanning equipment. Next time you are in the checkout line at a grocery store, check to see what type of scanning equipment they are using. Likely it will say Symbol on the housing.    Although we take bar codes for granted, there is a surprising amount of technology behind them. Bar codes exist primarily because conventional æAª optical character recognition (OCR) systems are not sufficiently reliable for inventory operations. The bar code symbology familiar to us on each box of cereal or pack of gum encodes a ten-digit number with sufficient error correction such that it is virtually impossible to scan the wrong number, even if the can is upside-down or dented. Occasionally, the cashier won't be able to get a label to scan at all, but once you hear that blip you know it was read correctly.yH _D1 0€‘€€çJ—K€‰€‚ÿThe ten-digit capacity of conventional bar code labels means that there is only room to store a single ID number in a label. Thus any application of supermarket bar codes must have a database mapping 11141-47011 to a particular size and brand of soy sauce. The holy grail of the bar code world has long been the development of higher-capacity bar code symbologies that can store entire documents, yet still be read reliably. Largely through the efforts of Theo Pavlidis and Ynjiun Wang at Stony Brook [PSW92], Symbol Technologies was ready to introduce the first such product.ïˆæANGg œ€€€†"€&€€‚ç}± ݀†"€‰€‚‚‚ç=ù#d€†"€‰€‚ÿ  Figure: A two-dimensional bar-code label of the Gettysburg Address using PDF-417  ``PDF-417 is our new, two-dimensional bar code symbology,'' Ynjiun explained. A sample label is shown in Figure .  ``How much data can you fit in a typical one-inch square label?'' I asked him.``It depends upon the level of error correction we use, but about 1,000 bytes. That's enough for a small text file or image,'' he said.``Interesting. You will probably want to use some data compression technique to maximize the amount of text you can store in a label.'' See Section for a discussion of standard data compression algorithms.   ¬u_DúI7 <€í€€†"€'€€‚‚ÿ  Figure: Mode switching in PDF-417  ``We do incorporate a data compaction method,'' he explained. ``We figure there are several different types of files our customers will want to use our labels for. Some files will be all in uppercase letters, while others will use mixed-case letters and numbers. We provide four different text modes in our code, each with a different subset of ASCII characters available. So long as we stay within a mode, we can describe each character using only five bits. When we have to switch modes, we issue a mode switch command first (taking an extra five bits) and then the new code.''  ÇNGÁLH ^€€€ç]Ï}$€†"€‰€‚€€€€‚ÿ``I see. So you designed the mode character sets to try to minimize the number of mode switch operations on typical text files.'' The modes are illustrated in Figure .``Right. We put all the digits in one mode and all the punctuation characters in another. We also included both mode shift and mode latch commands. In a mode shift, we switch into a new mode just for the next character, say to produce a punctuation mark. This way, we don't pay a cost for returning back to text mode after a period. Of course, we can also latch permanently into a different mode if we will be using a run of several characters from there.''~SúI?O+ $€§€€‚‚‚‚ÿ``Wow!'' I said. ``With all of this mode switching going on, there must be many different ways to encode any given text as a label. How do you find the smallest such encoding.''``We use a greedy-type algorithm. We look a few characters ahead and then decide which mode we would be best off in. It works fairly well.''I pressed him on this. ``How do you know it works fairly well? There might be significantly better encodings that you are simply not finding.''``I guess I don't know. But it's probably NP-complete to find the optimal coding.'' Ynjiun's voice trailed off. ``Isn't it?''|WÁLǁ% €¯€€‚ÿI started to think. Every encoding started in a given mode and consisted of a sequence of intermixed character codes and mode shift/latch operations. At a?Oǁª ny given position in the text, we could output the next character code (if it was available in our current mode) or decide to shift. As we moved from left to right through the text, our current state would be completely reflected by our current character position and current mode state. For a given position/mode pair, we would have been interested in the cheapest way of getting there, over all possible encodings getting to this point....6?OH„K d€q€€‚†"€(†"€ç€€€€€€‚ÿMy eyes lit up so bright they cast shadows on the walls.``The optimal encoding for any given text in PDF-417 can be found using dynamic programming. For each possible mode and each character position , we will maintain the cheapest encoding found of the string to the left of i ending in mode m. From each mode/position, we can either match, shift, or latch, so there are only few possible operations to consider. Each of the 4n cells can be filled in constant time, so it takes time linear in the length of the string to find the optimal encoding.''¯„ǁ÷‡+ $€ €€€€‚ÿYnjiun was skeptical, but he encouraged us to implement an optimal encoder. A few complications arose due to weirdnesses of PDF-417 mode switching, but my student Yaw-Ling Lin rose to the challenge. Symbol compared our encoder to theirs on 13,000 labels and concluded that dynamic programming lead to an 8% tighter encoding on average. This was significant, because no one wants to waste 8% of their potential storage capacity, particularly in an environment where the capacity is only a few hundred bytes. For certain applications, this 8% margin permitted one bar code label to suffice where previously two had been required. Of course, an 8% average improvement meant that it did much better than that on certain labels. While our encoder took slightly longer to run than the greedy encoder, this was not significant, since the bottleneck would be the time needed to print the label, anyway.“kH„Š‰( €×€€€‚ÿOur observed impact of replacing a heuristic solution with the global optimum is probably typical of most applications. Unless you really botch your heuristic, you are probably going to get a decent solution. But replacing it with an optimal result usually gives a small but non-trivial improvement, which can have pleasing consequences for your application.)÷‡³‰& €€€‚ÿʉv‹ Ё•€€‚çÐ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÎ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æs·Û/€†"€‰€æt·Û/€†"€‰€æu·Û/€†"€‰€€çÐ8ۀ‰€€çÿ7ۀ‰€€çÎ8ۀ‰€€‚‚ÿ Next: Divide and Conquer Up: Breaking Problems Down Previous: War Story: What's Past AlgorithmsMon Jun 2 23:33:50 EDT 1997)³‰Ÿ‹& €€€‚ÿµfv‹TŒO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Ÿ‹…Œ1Uÿÿÿÿÿÿÿÿyÿÿÿÿ…Œ©Œ$TŒ©Œ" €€€ ÿC…ŒìŒ1c ÿÿÿÿÿÿÿÿzìŒ%/ÇDivide and Conquer9©Œ%# €,€€‚ÿDivide and Conquer ?8ìŒpÀ ܁€€çÑ8ۀ†"€ ‰€çÿ7ۀ†"€ ‰€çÏ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æv·Û/€†"€‰€æw·Û/€†"€‰€æx·Û/€†"€‰€€çÑ8ۀ‰€€çÿ7ۀ‰€€çÏ8ۀ‰€‚€‚€‚‚ÿ Next: Fast Exponentiation Up: Breaking Problems Down Previous: War Story: Text CompressionDivide and Conquer Divide and conquer was a successful military strategy long before it became an algorithm design paradigm. Generals observed that it was easier to defeat one army of 50,000 men, followed by another army of 50,000 men than it was to beat a single 100,000 man army. Thus the wise general %pÀ©Œwould attack so as to divide the enemy army into two forces and then mop up one after the other.  e%ÕÂI `€=€€çgm€†"€‰€€€†"€‚ÿTo use divide and conquer as an algorithm design technique, we must divide the problem into two smaller subproblems, solve each of them recursively, and then meld the two partial solutions into one solution to the full problem. Whenever the merging takes less time than solving the two subproblems, we get an efficient algorithm. Mergesort, discussed in Section , is the classic example of a divide-and-conquer algorithm. It takes only linear time to merge two sorted lists of n/2 elements each of which was obtained in time.  êÂpÀ¿Ä( €…€€€‚ÿDivide and conquer is a design technique with many important algorithms to its credit, including mergesort, the fast Fourier transform, and Strassen's matrix multiplication algorithm. However, with the exception of binary search, I find it to be a difficult technique to apply in practice. Therefore, the examples below will illustrate binary search and its variants, which yield simple algorithms to solve a surprisingly wide range of problems.)ÕÂèÄ& €€€‚ÿ&¿ÄÅ# €€€‚ÿ¡QèįÅP p€¢€ȀçDÎ-›€‰€‚€ç¥\4ú€‰€‚€çë:Y€‰€‚ÿu Fast Exponentiation u Binary Search u Square and Other Roots&ÅÕÅ# €€€‚ÿ)¯ÅþÅ& €€€‚ÿS+ÕÅQÆ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)þÅzÆ& €€€‚ÿµfQÆ/ÇO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1zÆ`Ç1Uÿÿÿÿÿÿÿÿ{ÿÿÿÿ`Ç„Ç$/Ç„Ç" €€€ ÿD`ÇÈÇ1!ÿÿÿÿÿÿÿÿ|ÈÇÈ Fast Exponentiation:„ÇÈ# €.€€‚ÿFast Exponentiation vMÈÇxÊ) ‚¯€€çÒ8ۀ†"€ ‰€çÐ8ۀ†"€ ‰€çÐ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æy·Û/€†"€‰€æz·Û/€†"€‰€抷Û/€†"€‰€€çÒ8ۀ‰€€çÐ8ۀ‰€€çÐ8ۀ‰€‚€ ‚€†"€)€€ç !‹€†"€‰€‚ÿ Next: Binary Search Up: Divide and Conquer Previous: Divide and ConquerFast ExponentiationSuppose that we need to compute the value of for some reasonably large n. Such problems occur in primality testing for cryptography, as discussed in Section .  ­IÈ%Ìd –€€€€€†"€*†"€+€€†"€,€€†"€-†"€W‚ÿThe simplest algorithm performs n-1 multiplications, by computing . However, we can do better by observing that . If n is even, then . If n is odd, then . In either case, we have halved the size of our exponent at the cost of at most two multiplications, so multiplications suffice to compute the final value.s÷xʘÎ| ƀõ€€‚€€‚€€‚€€†"€.‚€€†"€/‚†"€0‚€€€€‚ÿ function power(a,n) if (n = 0) return(1) x = power( ) if (n is even) then return( ) else return( ) This simple algorithm illustrates an important principle of divide and conquer. It always pays to divide a job as evenly as possible. This principle applies to real life as well. When n is not a power of two, the problem cannot always be divided perfectly evenly, but a difference of one element between the two sides cannot cause any serious imbalance.)%ÌÁÎ& €€€‚ÿS+˜ÎÏ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)ÁÎ=Ï& €€€‚ÿµfÏ O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b=Ï „Ç1=Ï=1Uÿÿÿÿÿÿÿÿ}ÿÿÿÿ=a$ a" €€€ ÿ> =Ÿ1Æ ÿÿÿÿÿÿÿÿ~ŸÓhBinary Search4aÓ# €"€€‚ÿBinary Search ŠbŸyD V‚×€€çÓ8ۀ†"€ ‰€çÐ8ۀ†"€ ‰€çÑ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€杷Û/€†"€‰€枷Û/€†"€‰€柷Û/€†"€‰€€çÓ8ۀ‰€€çÐ8ۀ‰€€çÑ8ۀ‰€‚€ ‚€€€€€€€€€€€€€€€€€†"€1€€‚ÿ Next: Square and Other Roots Up: Divide and Conquer Previous: Fast ExponentiationBinary SearchBinary search is a fast algorithm for searching in a sorted array S of keys. To search for key q, we compare q to the middle key S[n/2]. If q appears before S[n/2], it must reside in the top half of our set; if not, it must reside in the bottom half of our set. By recursively repeating this process on the correct half, we find the key in a total of comparisons, a big win over the n/2 we expect with sequential search.  DÓœ+ $€3€€€€‚ÿThis much you probably know. What is important is to have a sense for just how fast binary search is. Twenty questions is a popular children's game, where one player selects a word, and the other repeatedly asks true/false questions in an attempt to identify the word. If the word remains unidentified after 20 questions, the first party wins; otherwise, the second player takes the honors. In fact, the second player always has a winning strategy, based on binary search. Given a printed dictionary, the player opens it in the middle, selects a word (say ``move''), and asks whether the unknown word is before ``move'' in alphabetical order. Since standard dictionaries contain 50,000 to 200,000 words, we can be certain that the process will always terminate within twenty questions.  ä}y¡ g œ€ý€€€€†"€1€€€€€€€€€€€€€€€€€‚ÿOther interesting algorithms follow from simple variants of binary search. For example, suppose we have an array A consisting of a run of 0's, followed by an unbounded run of 1's, and would like to identify the exact point of transition between them. Binary search on the array would provide the transition point in tests, if we had a bound n on the number of elements in the array. In the absence of such a bound, we can test repeatedly at larger intervals (A[1], A[2], A[4], A[8], A[16], ) until we find a first non-zero value.     Now we have a window containing the target and can proceed with binary search. This one-sided binary search finds the transition point p using at most comparisons, regardless of how large the array actally is. One-sided binary search is most useful whenever we are looking for a key that probably lies close to our current position.  )œÊ & €€€‚ÿÀ¿¡ Š  Ё€€‚çÓ8ۀ†"€ ‰€çÐ8ۀ†"€ ‰€çÑ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ ·Û/€†"€‰€æ¡·Û/€†"€‰€梷Û/€†"€‰€€çÓ8ۀ‰€€çÐ8ۀ‰€€çÑ8ۀ‰€€‚‚ÿ Next: Square and Other Roots Up: Divide and Conquer Previous: Fast Exponentiation AlgorithmsMon Jun 2 23:33:50 EDT 1997)Ê ³ & €€€‚ÿµfŠ hO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1³ ™1Uÿÿÿÿÿÿÿÿÿÿÿÿ™œ$hœ" €€€ ÿG™1b ÿÿÿÿÿÿÿÿ€AaJSquare and Other Roots=œA# €4€€‚ÿSquare and Other Roots ˆnÕA ‚çÔ8ۀ†"€ ‰€çÐ8ۀ†"€ ‰€çÒ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€棷Û/€†"€‰€怷Û/€†"€‰€楷Û/€†AÕAœ"€‰€€çÔ8ۀ‰€€çÐ8ۀ‰€€çÒ8ۀ‰€‚€ ‚€€€€€†"€2‚ÿ Next: Exercises Up: Divide and Conquer Previous: Binary SearchSquare and Other RootsThe square root of n is the number r such that . Square root computations are performed inside every pocket calculator, but it is instructive to develop an efficient algorithm to compute square roots.  À A•Dµ 8#€€†"€3€€€€€€€€€€€€€€€€†"€4€€†"€5€€€€€€†"€6€€€€€€†"€š†"€7‚ÿFirst, observe that the square root of must be at least 1 and at most n. Let l=1 and r=n. Consider the midpoint of this interval, m=(l+r)/2. How does compare to n? If , then the square root must be greater than m, so the algorithm repeats with l=m. If , then the square root must be less than m, so the algorithm repeats with r=m. Either way, we have halved the interval with only one comparison. Therefore, after only rounds we will have identified the square root to within .È1ÕA]G— ü€c€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿThis bisection method, as it is called in numerical analysis, can also be applied to the more general problem of finding the roots of an equation. We say that x is a root of the function f if f(x)=0. Suppose that we start with values l and r such that f(l) > 0 and f(r) < 0. If f is a continuous function, there must be a root between l and r. Depending upon the sign of f(m), where m=(l+r)/2, we can cut this window containing the root in half with each test and stop when our estimate becomes sufficiently accurate.  ª‚•DI( €€€€‚ÿRoot-finding algorithms that converge faster than binary search are known for both of these problems. Instead of always testing the midpoint of the interval, these algorithms interpolate to find a test point closer to the actual root. Still, binary search is simple, robust, and works as well as possible without additional information on the nature of the function to be computed.)]G0I& €€€‚ÿS+IƒI( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)0I¬I& €€€‚ÿµfƒIaJO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¬I’J1Uÿÿÿÿÿÿÿÿÿÿÿÿ’J¶J$aJ¶J" €€€ ÿ: ’JðJ1}.ÿÿÿÿÿÿÿÿ‚ðJ KGExercises0 ¶J K# €€€‚ÿExercises Ž¯ðJÔL ؁o€€çcn_ɀ†"€ ‰€çÿ7ۀ†"€ ‰€çÓ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÑ·Û/€†"€‰€æÈ·Û/€†"€‰€æÉ·Û/€†"€‰€€çcn_ɀ‰€€çÿ7ۀ‰€€çÓ8ۀ‰€‚€‚€‚ÿ Next: Implementation Challenges Up: Breaking Problems Down Previous: Square and Other RootsExercises ~) KRNU x€[€Ȁ€€†"€8†"€ç†"€9†"€:€€‚ÿ1. Consider the problem of storing n books on shelves in a library. The order of the books is fixed by the cataloging system and so cannot be rearranged. Therefore, we can speak of a book , where , that has a thickness and height . The length of each bookshelf at this library is L.ÆÔLt€P n€‘€Ȁ€€†"€;€€€€€€†"€8‚ÿSuppose all the books have the same height h (i.e. for all i, j) and the shelves are all separated by a distance of greater than h, so any book fits on any shelf. The greedy algorithm would fill the first shelf with as many books as we can until we get the smallest i such that does not fit, and then repeat with subsequent shelves. ShoRNt€¶Jw that the greedy algorithm always finds the optimal shelf placement, and analyze its time complexity.lGRNà% €€Ȁ‚ÿ2. (*) This is a generalization of the previous problem. Now consider the case where the height of the books is not constant, but we have the freedom to adjust the height of each shelf to that of the tallest book on the shelf. Thus the cost of a particular layout is the sum of the heights of the largest book on each shelf.1ýt€ƒ4 6€û€‘€€€€‚€€‚ÿu Give an example to show that the greedy algorithm of stuffing each shelf as full as possible does not always give the minimum overall height.u Give an algorithm for this problem, and analyze its time complexity. Hint: use dynamic programming.تàéƒ. *€W€Ȁ†"€<‚ÿ3. (*) Consider a city whose streets are defined by an grid. We are interested in walking from the upper left-hand corner of the grid to the lower right-hand corner.Mƒ6…G \€€Ȁ†"€<€€€€€€€€‚ÿUnfortunately, the city has bad neighborhoods, which are defined as intersections we do not want to walk in. We are given an matrix BAD, where BAD[i,j] = ``yes'' if and only if the intersection between streets i and j is somewhere we want to avoid.¢wéƒ؅+ &€î€Ȁ€€‚ÿ(a) Give an example of the contents of BAD such that there is no path across the grid avoiding bad neighborhoods.•d6…m†1 2€È€Ȁ€€€€‚ÿ(b) Give an O( X Y ) algorithm to find a path across the grid that avoids bad neighborhoods.¬@؅‰l Š€‡€Ȁ€€€€€€†"€=‚†"€<†"€<€€€€€€€€‚ÿ(c) Give an O( X Y ) algorithm to find the shortest path across the grid that avoids bad neighborhoods. You may assume that all blocks are of equal length. For partial credit, give an algorithm.4. (*) Consider the same situation as the previous problem. We have a city whose streets are defined by an grid. We are interested in walking from the upper left-hand corner of the grid to the lower right-hand corner. We are given an matrix BAD, where BAD[i,j] = ``yes'' if and only if the intersection between streets i and j is somewhere we want to avoid.0þm†IŠ2 2€ý€Ȁ€€€€‚ÿIf there were no bad neighborhoods to contend with, the shortest path across the grid would have length (X-1) + (Y-1) blocks, and indeed there would be many such paths across the grid. Each path would consist of only rightward and downward moves.ŽC‰׋K d€‡€Ȁ€€€€€€€€€€‚€€‚ÿGive an algorithm that takes the array BAD and returns the number of safe paths of length X+Y-2. For full credit, your algorithm must run in O( X Y ).5. (*) Given an array of n real numbers, consider the problem of finding the maximum sum in any contiguous subvector of the input. For example, in the array3IŠ Œ. ,€ €Ȁ†"€>‚ÿ÷׋'& €ï€Ȁ‚ÿthe maximum is achieved by summing the third through seventh elements, where 59+26+(-53)+58+97 = 187. When all numbers are positive, the entire array is the answer, while when all numbers are negative, the empty array maximizes the total at 0.Y Œ€ŽU x€€‘€€€€†"€‡‚€€†"€π€†"€‚ÿu Give a simple, clear, and correct -time algorithm to find the maximum contiguous subvector.u Now give a -time dynamic programming algorithm for this problem. To get partial credit, you may instead give a correct divide-and-conquer algorithm.K'ˏ4 6€1€Ȁ†"€?€€‚ÿ6. In the United States, coins are minted with denominations of 1, 5, 10, 25, and 50 cents. Now consider a country whose coins are minted with denominations of units. They seek an algorithm that will enable them to make change of n units using the minimum number of coins./€ŽÁ/ ,€€Ȁ†"€@‚ÿ(a) ˏÁ¶JThe greedy algorithm for making change repeatedly uses the biggest coin smaller than the amount to be changed until it is zero. Show that the greedy algorithm does not always give the minimum number of coins in a country whose denominations are .%ˏ£Ãx Ÿ€Q€Ȁ€€†"€?‚†"€?€€€€€€†"€@€€€€€€€€€€‚ÿ(b) Give an efficient algorithm that correctly determines the minimum number of coins needed to make change of n units using denominations . Analyze its running time.7. (*) In the United States, coins are minted with denominations of 1, 5, 10, 25, and 50 cents. Now consider a country whose coins are minted with denominations of units. They want to count how many distinct ways C(n) there are to make change of n units. For example, in a country whose denominations are , C(5) = 1, C(6) to C(9)=2, C(10)=3, and C(12)=4.3Á0ÅZ ‚€i€‘€€†"€@‚€€€€€€€€€€€€€€‚ÿ1. How many ways are there to make change of 20 units from ?2. Give an efficient algorithm to compute C(n), and analyze its complexity. (Hint: think in terms of computing C(n,d), the number of ways to make change of n units with highest denomination d. Be careful to avoid overcounting.)æ £ÃÇF Z€C€Ȁ†"€A€€€€€€€€‚ÿ8. (**) Consider the problem of examining a string of characters from an alphabet on k symbols, and a multiplication table over this alphabet, and deciding whether or not it is possible to parenthesize x in such a way that the value of the resulting expression is a, where a belongs to the alphabet. The multiplication table is neither commutative or associative, so the order of multiplication matters.y 0ŏÇl#š€õ(õ(õ(€€Ȁ‚ÿ€€Ȁ‚ÿ€€Ȃÿ€€Ȃÿÿÿabcv ÇÈi#¢€ ÿÿÿÿ€€Ȁ‚ÿ€€Ȃÿ€€Ȃÿ€€Ȃÿÿÿaaccv Ç{Èi#¢€ ÿÿÿÿ€€Ȁ‚ÿ€€Ȃÿ€€Ȃÿ€€Ȃÿÿÿbaab{ÈöÈl#š€ ÿÿÿÿ€€Ȁ‚ÿ€€Ȃÿ€€Ȃÿ€€Ȁ‚ÿÿÿcccc '{ÈÉ$ €€Ȁ‚ÿ šöÈ'Êb ’€Q€Ȁ€€€€€€€€€€€€€€€€€€€€‚ÿFor example, consider the following multiplication table and the string bbbba. Parenthesizing it (b(bb))(ba) gives a, but ((((bb)b)b)a) gives c.²ÉBÍi  €e€Ȁ€€€€‚€€€€€€€€€€€€€€€€€€‚ÿGive an algorithm, with time polynomial in n and k, to decide whether such a parenthesization exists for a given string, multiplication table, and goal element.9. (*) Consider the following data compression technique. We have a table of m text strings, each of length at most k. We want to encode a data string D of length n using as few text strings as possible. For example, if our table contains (a,ba,abab,b) and the data string is bababbaababa, the best way to encode it is (b,abab,ba,abab,a) - a total of five code words. Give an O(nmk) algorithm to find the length of the best encoding. You may assume that the string has an encoding in terms of the table.õ'Ê\Î% €ë€Ȁ‚ÿ10. A company database consists of 10,000 sorted names, 40% of whom are known as good customers and who together account for 60% of the accesses to the data base. There are two data structure options to consider for representing the database:@ BÍœÏ4 6€€‘€€€€‚€€‚ÿu Put all the names in a single array and use binary search.u Put the good customers in one array and the rest of them in a second array. Only if we do not find the query name on a binary search of the first array do we do a binary search of the second array.÷ \ΟW |€E€Ȁ‚€€€€€€†"€B€€†"€C€€‚ÿDemonstrateœÏŸ¶J which option gives better expected performance. Does this change if linear search on an unsorted array is used instead of binary search for both options?11. Suppose you are given an array A of n sorted numbers that has been circularly shifted k positions to the right. For example, is a sorted array that has been circularly shifted k=2 positions, while has been shifted k=4 positions.hûœÏm š€ù€‘€€€€€€€€€€‚€€€€€€†"€W€€€€€€‚ÿu Suppose you know what k is. Give an O(1) algorithm to find the largest number in A.u Suppose you do not know what k is. Give an algorithm to find the largest number in A. For partial credit, you may give an O(n) algorithm.C֟Jm š€¹€Ȁ€€†"€D†"€W€€†"€E†"€F†"€G†"€H€€‚ÿ12. (*) Suppose that you are given a sorted sequence of distinct integers . Give an algorithm to determine whether there exists an index i such at . For example, in , . In , there is no such i.,v' € €€€‚ÿ)JŸ& €€€‚ÿÊÉvi Ё£€€‚çcn_ɀ†"€ ‰€çÿ7ۀ†"€ ‰€çÓ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÊ·Û/€†"€‰€æË·Û/€†"€‰€æÌ·Û/€†"€‰€€çcn_ɀ‰€€çÿ7ۀ‰€€çÓ8ۀ‰€€‚‚ÿ Next: Implementation Challenges Up: Breaking Problems Down Previous: Square and Other Roots AlgorithmsMon Jun 2 23:33:50 EDT 1997)Ÿ’& €€€‚ÿµfiGO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1’x1Uÿÿÿÿÿÿÿÿƒÿÿÿÿxœ$Gœ" €€€ ÿExá1yÿÿÿÿÿÿÿÿ„ቅThe Friendship Graph;œ# €0€€‚ÿThe Friendship Graph ³œáÏ  üK€€ç9ۀ†"€ ‰€çö7ۀ†"€ ‰€çö7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÍ·Û/€†"€‰€æηÛ/€†"€‰€æÏ·Û/€†"€‰€€ç9ۀ‰€€çö7ۀ‰€€çö7ۀ‰€‚€‚€†"€I€€‚‚ÿ Next: Data Structures for Graphs Up: Graph Algorithms Previous: Graph AlgorithmsThe Friendship Graph  Figure: A portion of the friendship graph  To demonstrate the importance of proper modeling, let us consider a graph where the vertices are people, and there is an edge between two people if and only if they are friends. This graph is well-defined on any set of people, be they the people in your neighborhood, at your school or place of business, or even the entire world. There are many interesting aspects of people that are best understood as properties of this friendship graph.  »–Š % €-€€‚ÿWe use this opportunity to define important graph theory terminology. ``Talking the talk'' proves to be an important part of ``walking the walk''.n@Ï ø . ,€€€Ȁ€€€‚ÿu If I am your friend, does that mean you are my friend? ŒŠ T v€y€Ȁ€€€€€€€€€€€€‚€€€‚ÿA graph is undirected if edge (x,y) always implies (y,x). Otherwise, the graph is said to be directed. The ``heard-of'' graph is directed, since many famous people who I have heard of have never heard of me! The ``had-sex-with'' graph is presumably undirected, since the critical operation always requires a partner. I'd like to think that the ``friendship'' graph is always an undirected graph.       u Am I my own friend? Áø /AZ ‚€ƒ€Ȁ€€€€€€€€€€€€€€‚€€€‚ÿAn edge of the form (x,x) is said to be a loop. If x was y's friend several times over, we can model this relationship using multiedges, multipl/Aœe edges between the same pair of vertices. A graph is said to be simple if it contains no loops and no multiple edges. Simple graphs really are often simpler to work with in practice. Therefore, we might be better off if no one was their own friend.    u How close a friend are you? îŒC2 2€y€Ȁ€€€€‚ÿA graph is said to be weighted if each edge has an associated numerical attribute. We could model the strength of a friendship by associating each edge with an appropriate number, say from 0 (enemies) to 10 (blood brothers). The edges of a road network graph might be weighted with their length, drive-time, or speed limit, depending upon the application. A graph is said to be unweighted if all edges are assumed to be of equal weight.}H/AšC5 :€’€Ȁ†"€J€€‚ÿ  Figure: Mel Brooks is my father's sister's husband's cousin  g9CD. ,€r€Ȁ€€€‚ÿu Am I linked by some chain of friends to a star? w,šCxEK d€[€Ȁ€€çM逆"€‰€‚€€€‚ÿA path is a sequence of edges connecting two vertices. Since Mel Brooks is my father's sister's husband's cousin, there is a path in the friendship graph between me and him, shown in Figure . This is true even though the two of us have never met.  u How close is my link to that star? ’fD H, &€Í€Ȁ€€‚ÿIf I were trying to impress you with how tight I am with Mel Brooks, I would be much better off saying that my Uncle Lenny grew up with him than to go into the details of how connected I am to Uncle Lenny. Through Uncle Lenny, I have a path of length 2 to Cousin Mel, while the path is of length 4 by blood and marriage. I could make the path even longer by linking in people who know both me and my father, or are friends of Aunt Eve and Uncle Lenny. This multiplicity of paths hints at why finding the shortest path between two nodes is important and instructive, even in non-transportation applications.  sIxE}H* $€’€Ȁ€€‚ÿu Is there a path of friends between every two people in the world? g5 HäI2 2€k€Ȁ€€€€‚ÿThe ``six degrees of separation'' theory argues that there is always a short path linking every two people in the world. We say that a graph is connected if there is a path between any two vertices. A directed graph is strongly connected if there is always a directed path between any two vertices.   Ÿ}HóKQ p€€Ȁ€€çÀH€†"€‰€€€‚€€€‚ÿIf a graph is not connected, we call each connected piece a connected component. If we envision tribes in remote parts of the world that have yet not been encountered, each such tribe would form a connected component in the friendship graph. A remote hermit, or extremely unpleasant fellow (see Figure ) would represent a connected component of one vertex, or an isolated vertex.  u Who has the most friends? The fewest friends? ÊäIûM> J€•€Ȁ€€€€€€€€‚ÿThe degree of a vertex is the number of edges adjacent to it. The most popular person defines the vertex of highest degree in the friendship graph. Remote hermits will be associated with degree-zero vertices. In dense graphs, most vertices have high degree, as opposed to sparse graphs with relatively few edges. In regular graphs, each vertex has exactly the same degree. A regular friendship graph would truly be the ultimate in social-ism.    M#óKHN* $€F€Ȁ€€‚ÿu What is the largest clique? ç±ûM;€6 :€c€Ȁ€€‚€€€‚ÿA social clique is a group of mutual friends who all hang around together. A graph-theoretic clique is a complete subgraph, where each vertex pair has an edge between them. Cliques are the densest possible subgraphs. Within the friendship graph, we would expect to see large cliques corresponding to workplaces, neighborhoods, religious organizations, and schools.  u How long wiHN;€œll it take for my gossip to get back to me? \ HN—‚P n€€Ȁ€€€€€€€€€€€€€€‚ÿA cycle is a path where the last vertex is adjacent to the first. A cycle in which no vertex repeats (such as 1-2-3-1 verus 1-2-3-2-1) is said to be simple. The shortest cycle in the graph defines the graph's girth, while a simple cycle that passes through every vertex once is said to be a Hamiltonian cycle. An undirected graph with no cycles is said to be a tree if it is connected; otherwise it is a forest. A directed graph with no directed cycles is said to be a DAG, or directed acyclic graph.      ,;€Â' € €€€‚ÿ)—‚ì‚& €€€‚ÿ¿ŸÃ‚«„ Ё€€‚ç9ۀ†"€ ‰€çö7ۀ†"€ ‰€çö7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æзÛ/€†"€‰€æü·Û/€†"€‰€æó·Û/€†"€‰€€ç9ۀ‰€€çö7ۀ‰€€çö7ۀ‰€€‚‚ÿ Next: Data Structures for Graphs Up: Graph Algorithms Previous: Graph Algorithms AlgorithmsMon Jun 2 23:33:50 EDT 1997)ì‚Ԅ& €€€‚ÿµf«„‰…O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Ԅº…1Uÿÿÿÿÿÿÿÿ…ÿÿÿÿº…ޅ$‰…ޅ" €€€ ÿKº…)†12ÿÿÿÿÿÿÿÿ†)†j†YÎData Structures for GraphsAޅj†# €<€€‚ÿData Structures for Graphs ©)†‰ ‚-€€çø8ۀ†"€ ‰€çö7ۀ†"€ ‰€çÕ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æô·Û/€†"€‰€æõ·Û/€†"€‰€æö·Û/€†"€‰€€çø8ۀ‰€€çö7ۀ‰€€çÕ8ۀ‰€‚€‚€‚çn€hl€†"€‰€‚ÿ Next: War Story: Getting the Up: Graph Algorithms Previous: The Friendship GraphData Structures for Graphs Selecting the right data structure to represent graphs can have an enormous impact on the performance of an algorithm. Your two basic choices are adjacency matrices and adjacency lists, illustrated in Figure .çkj†ú‹| ƀÝ€€€€†"€K€€€€€€€€€€€€€€€€€€†"€†"€L‚ÿAn adjacency matrix is an matrix M where (typically) M[i,j] = 1 if there is an edge from vertex i to vertex j and M[i,j]=0 if there is not. Adjacency matrices are the simplest way to represent graphs. However, they doom you to using space no matter how many edges are in the graph. For large graphs, this will kill you. Remember that 1,000,000, and work up from there. Although there is some potential for saving space by packing multiple bits per word or simulating a triangular matrix for undirected graphs, these cost some of the simplicity that makes adjacency matrices so appealing.  T¬‰ZÁš c €€€€‚†"€M€€€€‚€€€€€€€€€€€€€€€€†"€N†"€O€€†"€P†"€O€€€‚ÿ  Figure: The adjacency matrix and adjacency list of a given graph    Beyond simplicity, there are certain algorithmic operations that prove faster on adjacency matrices than adjacency lists. In particular, it takes time to test whether edge (i,j) is in a graph represented by an adjacency matrix. All we must do is read the appropriate bit.An adjacency list consists of an n-element array of pointers, where the ith element points to a linked list of the edges incident on vertex i. To test whether edge (i,j) is in the graph, we search the ith list for j. This takes , where is the degree of the ith vertex. For a complete or almost complete graph, , so testing the existence of an edge can be very expensive relative to adjacency matrices. However, can be much less than n when the graph is sparse. Most of the graphú‹ZÁޅs that one encounters in real life tend to be sparse. Recall the friendship graph as an example. Further, a surprising number of the most efficient graph algorithms can be and have been designed to avoid such edge-existence queries. The key is processing the edges in a systematic order like breadth-first or depth-first search.   o/ú‹ÉÃ@ N€_€€€€€€€€€€€‚ÿFor most applications, adjacency lists are the right way to go. The main drawback is the complexity of dealing with linked list structures. Things can be made arbitrarily hairy by adding extra pointers for special purposes. For example, the two versions of each edge in an undirected graph, (i,j) and (j,i), can be linked together by a pointer to facilitate deletions. Also, depending upon the operations you will perform on each list, you may or may not want it to be doubly linked, so that you can move backwards as easily as you move forwards.ÛZÁÞÅ: B€¹€€çƒ A€†"€‰€‚ÿIt is a good idea to use a well-designed graph data type as a model for building your own, or even better as the foundation for your application. We recommend LEDA (see Section ) as the best-designed general-purpose graph data structure currently available. It may be more powerful (and hence somewhat slower/larger) than what you need, but it does so many things right that you are likely to lose most of the potential do-it-yourself benefits through clumsiness.   yUÉÃWÆ$ €ª€€‚ÿIn summary, we have the following tradeoffs between adjacency lists and matrices:tÞÅËÆ\#ˆ€0ÿ?ÿ?€€ÿ€€€‚ÿ€€‚ÿ€ €‚ÿÿÿComparisonWinner­DWÆxÇi#¢€ˆÿ?ÿ?€€€‚ÿ(€€€€€€‚ÿ€\€‚ÿ€`€‚ÿÿÿFaster to test if (x, y) is in graph?adjacency matrices›>ËÆÈ]#Š€|ÿ?ÿ?€€€‚ÿ€€‚ÿ€V€‚ÿ€Z€‚ÿÿÿFaster to find the degree of a vertex?adjacency lists¶DxÇÉÈr#Ž€Šÿ?ÿ?€€€‚ÿ€€‚ÿ€B€‚ÿ:€F€€€€€†"€Q‚ÿÿÿLess memory on small graphs?adjacency lists (m+n) vs.  CÈiÉ]#Š€†ÿ?ÿ?€€€‚ÿ€€‚ÿ€>€‚ÿ€B€‚ÿÿÿLess memory on big graphs?adjacency matrices (a small win)¹JÉÈ"Êo#®€”ÿ?ÿ?€€€‚ÿ€€‚ÿ€@€‚ÿ4€D€€€€€€€‚ÿÿÿEdge insertion or deletion?adjacency matrices O(1) vs. O(d)¬=iÉÎÊo#®€~ÿ?ÿ?€€€‚ÿ€€‚ÿ€D€‚ÿ4€H€†"€R†"€‡‚ÿÿÿFaster to traverse the graph?adjacency lists vs. Ÿ4"ÊmËk#Š€hÿ?ÿ?€€€‚ÿ€€‚ÿ€<€‚ÿ€@€‚ÿ€b€‚ÿÿÿBetter for most problems?adjacency lists&ÎÊ“Ë# €€€‚ÿ)mËŒË& €€€‚ÿ¿Ÿ“Ë{Í Ё€€‚çø8ۀ†"€ ‰€çö7ۀ†"€ ‰€çÕ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ÷·Û/€†"€‰€æø·Û/€†"€‰€æù·Û/€†"€‰€€çø8ۀ‰€€çö7ۀ‰€€çÕ8ۀ‰€€‚‚ÿ Next: War Story: Getting the Up: Graph Algorithms Previous: The Friendship Graph AlgorithmsMon Jun 2 23:33:50 EDT 1997)ŒË€Í& €€€‚ÿµf{ÍYÎO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1€ÍŠÎ1Uÿÿÿÿÿÿÿÿ‡ÿÿÿÿŠÎ®Î$YήÎ" €€€ ÿMŠÎûÎ1NÿÿÿÿÿÿÿÿˆûÎ>Ï(JWar Story: Getting the GraphC ®Î>Ï# €@€€‚ÿWar Story: Getting the Graph A.û΋ ôm€€çù8ۀ†"€ ‰€çö7ۀ†"€ ‰€ç9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æú·Û/€†"€‰€æû·Û/€†"€‰€æ'žÛ/€†"€>Ï‹®Î‰€€çù8ۀ‰€€çö7ۀ‰€€ç9ۀ‰€‚€‚€‚€€€€‚ÿ Next: Traversing a Graph Up: Graph Algorithms Previous: Data Structures for GraphsWar Story: Getting the Graph ``It takes five minutes just to read the data. We will never have time to make it do something interesting.''  Ì]>ÏWo ¬€Á€€‚†"€S€€‚çÕ«4€€†"€‰€€€€€ç-€†"€‰€‚ÿThe young graduate student was bright and eager, but green to the power of data structures. She would soon come to appreciate the power.  Figure: The dual graph (dashed lines) of a triangulation  As described in a previous war story (see Section ), we were experimenting with algorithms for extracting triangular strips for the fast rendering of triangulated surfaces. The task of finding a small number of strips that cover each triangle in a mesh could be modeled as a graph problem, where the graph has a vertex for every triangle of the mesh, and there is an edge between every pair of vertices representing adjacent triangles. This dual graph representation of the planar subdivision representing the triangulation (see Figure ) captures all the information about the triangulation needed to partition it into triangle strips.   Ü³‹3) €g€€‚‚‚ÿThe first step in crafting a program that constructs a good set of strips was to build the dual graph of the triangulation. This I sent the student off to do. A few days later, she came back and announced that it took over five CPU minutes just to construct this dual graph of an object with a few thousand triangles.``Nonsense!'' I proclaimed. ``You must be doing something very wasteful in building the graph. What format is the data in?''``Well, it starts out with a list of the 3D-coordinates of the vertices used in the model and then follows with a list of triangles. Each triangle is described by a list of three indices into the vertex coordinates. Here is a small example:''f:W™ , &€u€€‚ÿ VERTICES 4 0.000000 240.000000 0.000000 204.000000 240.000000 0.000000 204.000000 0.000000 0.000000 0.000000 0.000000 0.000000 TRIANGLES 2 0 1 3 1 2 3;3Ô + $€!€€‚‚‚‚ÿ``I see. So the first triangle must use all but the third point, since all the indices start from zero. The two triangles must share an edge formed by points 1 and 3.''``Yeah, that's right,'' she confirmed.``OK. Now tell me how you built your dual graph from this file.''``Well, I can pretty much ignore the vertex information, once I know how many vertices there will be. The geometric position of the points doesn't affect the structure of the graph. My dual graph is going to have as many vertices as the number of triangles. I set up an adjacency list data structure with that many vertices. As I read in each triangle, I compare it to each of the others to check whether it has two numbers in common. Whenever it does, I add an edge from the new triangle to this one.''á™ í8 >€Å€€€€‚‚†"€‚ÿI started to sputter. ``But that's your problem right there! You are comparing each triangle against every other triangle, so that constructing the dual graph will be quadratic in the number of triangles. Reading in the input graph should take linear time!''``I'm not comparing every triangle against every other triangle. On average, it only tests against half or a third of the triangles.''``Swell. But that still leaves us with an algorithm. That is much too slow.''m"Ô fAK d€E€€‚€€€€€€€€€€€€‚ÿShe stood her ground. ``Well, don't just complain. Help me fix it!''Fair enough. I started to think. We needed some quick method to screen away most of the triangles that would not be adjacent to ífA®Îthe new triangle (i,j,k). What we really needed was just a list of all the triangles that go through each of the points i, j, and k. Since each triangle goes through three points, the average point is incident on three triangles, so this would mean comparing each new triangle against fewer than ten others, instead of most of them.Ÿ™í$D% €3€€‚ÿ``We are going to need a data structure consisting of an array with one element for every vertex in the original data set. This element is going to be a list of all the triangles that pass through that vertex. When we read in a new triangle, we will look up the three relevant lists in the array and compare each of these against the new triangle. Actually, only two of the three lists are needed, since any adjacent triangle will share two points in common. For anything sharing two vertices, we will add an adjacency to our graph. Finally, we will add our new triangle to each of the three affected lists, so they will be updated for the next triangle read.''<ûfA`GA P€ù€€‚çÕ«4€€†"€‰€‚€‚ÿShe thought about this for a while and smiled. ``Got it, Chief. I'll let you know what happens.''The next day she reported that the graph could be built in seconds, even for much larger models. From here, she went on to build a successful program for extracting triangle strips, as reported in Section .The take-home lesson here is that even elementary problems like initializing data structures can prove to be bottlenecks in algorithm development. Indeed, most programs working with large amounts of data have to run in linear or almost linear time. With such tight performance demands, there is no room to be sloppy. Once you focus on the need for linear-time performance, an appropriate algorithm or heuristic can usually be found to do the job.)$D‰G& €€€‚ÿÁÀ`GJI Ё‘€€‚çù8ۀ†"€ ‰€çö7ۀ†"€ ‰€ç9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æžÛ/€†"€‰€æžÛ/€†"€‰€æ žÛ/€†"€‰€€çù8ۀ‰€€çö7ۀ‰€€ç9ۀ‰€€‚‚ÿ Next: Traversing a Graph Up: Graph Algorithms Previous: Data Structures for Graphs AlgorithmsMon Jun 2 23:33:50 EDT 1997)‰GsI& €€€‚ÿµfJI(JO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1sIYJ1Uÿÿÿÿÿÿÿÿ‰ÿÿÿÿYJ}J$(J}J" €€€ ÿCYJÀJ1ÿÿÿÿÿÿÿÿŠÀJùJ©ÂTraversing a Graph9}JùJ# €,€€‚ÿTraversing a Graph »ŽÀJŽM ܁y€€çú8ۀ†"€ ‰€çö7ۀ†"€ ‰€çø8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ!žÛ/€†"€‰€æ"žÛ/€†"€‰€æ#žÛ/€†"€‰€€çú8ۀ‰€€çö7ۀ‰€€çø8ۀ‰€‚€‚€‚‚ÿ Next: Breadth-First Search Up: Graph Algorithms Previous: War Story: Getting theTraversing a Graph Perhaps the most fundamental graph problem is to traverse every edge and vertex in a graph in a systematic way. Indeed, most of the basic algorithms you will need for bookkeeping operations on graphs will be applications of graph traversal. These include:      7ùJ3OH ^€o€Ȁ€€‚€€‚€€‚€€‚€€‚ÿu Printing or validating the contents of each edge and/or vertex.u Copying a graph, or converting between alternate representations.u Counting the number of edges and/or vertices.u Identifying the connected components of the graph.u Finding paths between two vertices, or cycles if they exist.NŽMŸ1 0€€€€€€€‚ÿSince any maze can be represented by a graph, where each junction is a vertex and each hallway an edge, any traversal algorithm must be powerful enough to3OŸ}J get us out of an arbitrary maze. For efficiency, we must make sure we don't get lost in the maze and visit the same place repeatedly. By being careful, we can arrange to visit each edge exactly twice. For correctness, we must do the traversal in a systematic way to ensure that we don't miss anything. To guarantee that we get out of the maze, we must make sure our search takes us through every edge and vertex in the graph.  Še3OHƒ% €Ë€€‚ÿThe key idea behind graph traversal is to mark each vertex when we first visit it and keep track of what we have not yet completely explored. Although bread crumbs or unraveled threads are used to mark visited places in fairy-tale mazes, we will rely on Boolean flags or enumerated types. Each vertex will always be in one of the following three states:IŸ‘„C T€ €Ȁ€€€‚€€€‚€€€‚ÿu undiscovered - the vertex in its initial, virgin state.u discovered - the vertex after we have encountered it, but before we have checked out all its incident edges.u completely-explored - the vertex after we have visited all its incident edges.äHƒuˆW |€€€€€€€€€€€‚€€€€€€€€‚ÿObviously, a vertex cannot be completely-explored before we discover it, so over the course of the traversal the state of each vertex progresses from undiscovered to discovered to completely-explored.We must also maintain a structure containing all the vertices that we have discovered but not yet completely explored. Initially, only a single start vertex is considered to have been discovered. To completely explore a vertex, we must evaluate each edge going out of it. If an edge goes to an undiscovered vertex, we mark it discovered and add it to the list of work to do. If an edge goes to a completely-explored vertex, we will ignore it, since further contemplation will tell us nothing new about the graph. We can also ignore any edge going to a discovered but not completely-explored vertex, since the destination must already reside on the list of vertices to completely explore.¬o‘„!‹= H€ß€€€€€€€€€€‚ÿRegardless of which order we use to fetch the next vertex to explore, each undirected edge will be considered exactly twice, once when each of its endpoints is explored. Directed edges will be consider only once, when exploring the source vertex. Every edge and vertex in the connected component must eventually be visited. Why? Suppose the traversal didn't visit everything, meaning that there exists a vertex u that remains unvisited whose neighbor v was visited. This neighbor v will eventually be explored, and we will certainly visit u when we do so. Thus we must find everything that is there to be found.÷ÆuˆŒ1 0€€€€€€€‚ÿThe order in which we explore the vertices depends upon the container data structure used to store the discovered but not completely-explored vertices. There are two important possibilities:ŸZ!‹·ŽE X€µ€Ȁ€€€€€‚€€€€€‚ÿu Queue - by storing the vertices in a first in, first out (FIFO) queue, we explore the oldest unexplored vertices first. Thus our explorations radiate out slowly from the starting vertex, defining a so-called breadth-first search.  u Stack - by storing the vertices in a last in, first out (LIFO) stack, we explore the vertices by lurching along a path, visiting a new neighbor if one is available, and backing up only when we are surrounded by previously discovered vertices. Thus our explorations quickly wander away from our starting point, defining a so-called depth-first search.  &Œݎ# €€€‚ÿ)·Ž& €€€‚ÿ&ݎ,# €€€‚ÿx8€@ P€p€Ȁç͛:Ӏ‰€‚€ç.*A2€‰€‚ÿu Breadth-First Search u Depth-First Search&,ʏ# €€€‚ÿ)€ À& €€€‚ÿʏ À}J¿ŸÊËÁ Ё€€‚çú8ۀ†"€ ‰€çö7ۀ†"€ ‰€çø8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ$žÛ/€†"€‰€æ%žÛ/€†"€‰€æ&žÛ/€†"€‰€€çú8ۀ‰€€çö7ۀ‰€€çø8ۀ‰€€‚‚ÿ Next: Breadth-First Search Up: Graph Algorithms Previous: War Story: Getting the AlgorithmsMon Jun 2 23:33:50 EDT 1997) ÀôÁ& €€€‚ÿµfËÁ©ÂO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ôÁÚÂ1Uÿÿÿÿÿÿÿÿ‹ÿÿÿÿÚÂþÂ$©ÂþÂ" €€€ ÿEÚÂCÃ1ØÿÿÿÿÿÿÿÿŒCÃ~ìBreadth-First Search;þÂ~Ã# €0€€‚ÿBreadth-First Search ðCÊøó€€çû8ۀ†"€ ‰€çù8ۀ†"€ ‰€çù8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æRžÛ/€†"€‰€æIžÛ/€†"€‰€æJžÛ/€†"€‰€€çû8ۀ‰€€çù8ۀ‰€€çù8ۀ‰€‚€ ‚€†"€T€€‚ÿ Next: Depth-First Search Up: Traversing a Graph Previous: Traversing a GraphBreadth-First Search  Figure: An undirected graph and its breadth-first search tree  òŽ~ÃuÈd –€€€€€€€€€€€€€€€€€çś:Ӏ†"€‰€‚ÿThe basic breadth-first search algorithm is given below. At some point during the traversal, every node in the graph changes state from undiscovered to discovered. In a breadth-first search of an undirected graph, we assign a direction to each edge, from the discoverer u to the discovered v. We thus denote u to be the parent p[v]. Since each node has exactly one parent, except for the root, this defines a tree on the vertices of the graph. This tree, illustrated in Figure , defines a shortest path from the root to every other node in the tree. This property makes breadth-first search very useful in shortest path problems.   QƒÅ•ÊÏ l«€€‚€€‚†"€U‚€€€€‚€€€€€€‚€€‚€€€€‚†"€V‚†"€W‚€€€€‚€€‚†"€X‚€€€€‚ÿ BFS(G,s) for each vertex do state[u] = ``undiscovered'' p[u] = nil, i.e. no parent is in the BFS tree state[] = ``discovered'' p[] = nil while do u = dequeue[Q] process vertex u as desired for each do process edge (u,v) as desiredŒuȮ͍ 耀€€€€€‚€€€€‚€€€€€€‚€€‚€€€€‚€†"€Y€€€€‚ÿ if state[v] = ``undiscovered'' then state[v] = ``discovered'' p[v] = u enqueue[Q,v] state[u] = ``completely-explored'' The graph edges that do not appear in the breadth-first search tree also have special properties. For undirected graphs, non-tree edges can point only to vertices on the same level as the parent vertex or to vertices on the level directly below the parent. These properties follow easily from the fact that each path in the tree must be the shortest path in the graph. For a directed graph, a back-pointing edge can exist whenever v lies closer to the root than u does.0•ÊÞÎ( €€€€‚ÿThe breadth-first search algorithm above includes places to optionally process each vertex and edge, say to copy them, print them, or count them. Each vertex and directed edge is encountered exactly once, and each undirected edge is encountered exactly twice.)®ÍÏ& €€€‚ÿ»ºÞÎÎ Ё…€€‚çû8ۀ†"€ ‰€çù8ۀ†"€ ‰€çù8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æKžÛ/€†"€‰€æLžÛ/€†"€‰€æMžÛ/€†"€‰€€çû8ۀ‰€€çù8ۀ‰€€çù8ۀ‰€ÏÎþ€‚‚ÿ Next: Depth-First Search Up: Traversing a Graph Previous: Traversing a Graph AlgorithmsMon Jun 2 23:33:50 EDT 1997)Ï÷& €€€‚ÿµfάO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1÷Ý1UÿÿÿÿÿÿÿÿÿÿÿÿÝ$¬" €€€ ÿCÝD1ÐÿÿÿÿÿÿÿÿŽD} BDepth-First Search9}# €,€€‚ÿDepth-First Search ûD ø €€çü8ۀ†"€ ‰€çù8ۀ†"€ ‰€çú8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æNžÛ/€†"€‰€æOžÛ/€†"€‰€æPžÛ/€†"€‰€€çü8ۀ‰€€çù8ۀ‰€€çú8ۀ‰€‚€ ‚€†"€Z€€‚ÿ Next: Applications of Graph Traversal Up: Traversing a Graph Previous: Breadth-First SearchDepth-First Search  Figure: An undirected graph and its depth-first search tree  éV}v “ ô€³€€ç&*A2€†"€‰€‚†"€[€€‚ç)*A2€†"€‰€€€€€€€€€€€€€€€€€‚ÿDepth-first search turns out to be, in general, even more useful than breadth-first search. The reason is that a depth-first search of a graph organizes the edges of the graph in a very precise way, which is quite different from breadth-first search. As with BFS, we assign a direction to each edge when we discover it, as shown in Figure .    Figure: Edge possibilities for search trees  Although there are four conceivable classes of edges resulting from such labelings, as shown in Figure , only two of them can occur with undirected graphs. In a DFS of an undirected graph, every edge is either in the tree or goes directly back to an ancestor. Why? Suppose we encountered a forward edge (x,y) directed toward a decendant vertex. In this case, we would have discovered (x,y) when exploring y, making it a back edge. Suppose we encounter a cross edge (x,y), linking two unrelated vertices. Again, we would have discovered this edge when we explored y, making it a tree edge. For directed graphs, depth-first search labelings can take on a wider range of possibilities.”p $ €à€€‚ÿDepth-first search has a neat recursive implementation, which eliminates the need to explicitly use a stack:Êv Ô ­ (=€€‚€€‚€€€€‚€€‚†"€X‚€€€€‚€€€€‚€€€€€€‚€€‚€€€€‚€‚ÿ DFS(G,u) state[u] = ``discovered'' process vertex u if desired for each do process edge (u,v) if desired if state[v] = ``undiscovered'' then p[v] = u DFS(G,v) state[u] = ``completely-explored'' As with BFS, this implementation of the depth-first search algorithm includes places to optionally process each vertex and edge, say to copy them, print them, or count them. Both algorithms will traverse all edges in the same connected component as the starting point. Since we need to start with a vertex in each component in order to traverse a disconnected graph, we must start from any vertex remaining undiscovered after a component search. With the proper initialization, this completes the traversal algorithm:[Ý /~ ʀ¿€€‚€€‚†"€\‚€€€€‚†"€\‚€€€€‚‚€€‚€‚ÿ DFS-graph(G) for each vertex do state[u] = ``undiscovered'' for each vertex do if state[u] = ``undiscovered'' then initialize new component, if desired DFS[G,u] )Ô X& €€€‚ÿÊÉ/.A Ё£€€‚çü8ۀ†"€ ‰€çù8ۀ†"€ ‰€çú8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æQžÛ/€†"€‰€æ}žÛ/€X.A†"€‰€ætžÛ/€†"€‰€€çü8ۀ‰€€çù8ۀ‰€€çú8ۀ‰€€‚‚ÿ Next: Applications of Graph Traversal Up: Traversing a Graph Previous: Breadth-First Search AlgorithmsMon Jun 2 23:33:50 EDT 1997)XWA& €€€‚ÿµf.A BO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1WA=B1Uÿÿÿÿÿÿÿÿÿÿÿÿ=BaB$ BaB" €€€ ÿP=B±B11ÿÿÿÿÿÿÿÿ±B÷B÷IApplications of Graph TraversalF#aB÷B# €F€€‚ÿApplications of Graph Traversal ,±B#G& ‚€€çý8ۀ†"€ ‰€çö7ۀ†"€ ‰€çû8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æužÛ/€†"€‰€ævžÛ/€†"€‰€æwžÛ/€†"€‰€€çý8ۀ‰€€çö7ۀ‰€€çû8ۀ‰€‚€‚€€€€€€€€€€€€‚ÿ Next: Connected Components Up: Graph Algorithms Previous: Depth-First SearchApplications of Graph TraversalMost elementary graph algorithms involve making one or two traversals of the graph, while we update our knowledge of the graph as we visit each edge and vertex. Properly implemented using adjacency lists, any such algorithm is destined to be very fast. Both BFS and DFS run in O(n+m) on both directed and undirected graphs where, as usual, n is the number of vertices and m the number of edges in the graph. This is optimal, since it is as fast as one can hope to read the graph. The trick is seeing when traversal approaches are destined to work. We present several examples below.)÷BLG& €€€‚ÿ&#GrG# €€€‚ÿ”LGwHq °€)€ȀçðFNð€‰€‚€çQÕTO€‰€‚€ç²c[®€‰€‚€çòa €‰€‚€ç¿j‚b€‰€‚ÿu Connected Components u Tree and Cycle Detection u Two-Coloring Graphs u Topological Sorting u Articulation Vertices&rGH# €€€‚ÿ)wHÆH& €€€‚ÿS+HI( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)ÆHBI& €€€‚ÿµfI÷IO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1BI(J1Uÿÿÿÿÿÿÿÿ‘ÿÿÿÿ(JLJ$÷ILJ" €€€ ÿE(J‘J1]ÿÿÿÿÿÿÿÿ’‘JÌJ €Connected Components;LJÌJ# €0€€‚ÿConnected Components +&‘J÷M ؁]€€çþ8ۀ†"€ ‰€çü8ۀ†"€ ‰€çü8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æxžÛ/€†"€‰€æyžÛ/€†"€‰€æzžÛ/€†"€‰€€çþ8ۀ‰€€çü8ۀ‰€€çü8ۀ‰€‚€ ‚€‚ÿ Next: Tree and Cycle Detection Up: Applications of Graph Traversal Previous: Applications of Graph TraversalConnected ComponentsEither breadth-first or depth-first search can be used to identify the connected components of an undirected graph and label each vertex with the identifier of its components. In particular, we can modify the DFS-graph algorithm to increment a counter for the current component number and label each vertex accordingly as it is discovered in DFS.  CôÌJ:OO l€ë€€€€€€€€ç±Db»€†"€‰€€‚ÿFor directed graphs, there are two distinct notions of connectivity, leading to algorithms for finding both weakly connected and strongly connected components. Both of these can be found in O(n+m) time, as discussed in Section .)÷McO& €€€‚ÿS+:O¶O( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)cOßO& €€€‚ÿµf¶O €O n€Ì€€ßO €LJÌ#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ßOр1Uÿÿÿÿÿÿÿÿ“ÿÿÿÿрõ€$ €õ€" €€€ ÿIр>1Œÿÿÿÿÿÿÿÿ”>}؇Tree and Cycle Detection?õ€}# €8€€‚ÿTree and Cycle Detection ”ƒ>… ð€€çÿ8ۀ†"€ ‰€çü8ۀ†"€ ‰€çý8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ{žÛ/€†"€‰€æ|žÛ/€†"€‰€æšžÛ/€†"€‰€€çÿ8ۀ‰€€çü8ۀ‰€€çý8ۀ‰€‚€ ‚€€€€€‚ÿ Next: Two-Coloring Graphs Up: Applications of Graph Traversal Previous: Connected ComponentsTree and Cycle DetectionTrees are connected, undirected graphs that do not contain cycles. They are perhaps the simplest interesting class of graphs. Testing whether a graph is a tree is straightforward using depth-first search. During search, every edge will be labeled either a tree edge or a back edge, so the graph is a tree if and only if there are no back edges. Since m=n-1 for any tree, this algorithm can be said to run in time linear in the number of vertices.   m}~†X ~€+€€€€€€€€€€€€€€€€€€€‚ÿIf the graph is not a tree, it must contain a cycle. Such a cycle can be identified as soon as the first back edge (u,v) is detected. If (u,v) is a back edge, then there must be a path in the tree from v to u. Coupled with edge (u,v), this defines a cycle.)…§†& €€€‚ÿS+~†ú†( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)§†#‡& €€€‚ÿµfú†؇O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1#‡ ˆ1Uÿÿÿÿÿÿÿÿ•ÿÿÿÿ ˆ-ˆ$؇-ˆ" €€€ ÿD ˆqˆ1Ò ÿÿÿÿÿÿÿÿ–qˆ«ˆµÄTwo-Coloring Graphs:-ˆ«ˆ# €.€€‚ÿTwo-Coloring Graphs Š€qˆQŒ& ‚€€ç9ۀ†"€ ‰€çü8ۀ†"€ ‰€çþ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€柞Û/€†"€‰€æ žÛ/€†"€‰€æ¡žÛ/€†"€‰€€ç9ۀ‰€€çü8ۀ‰€€çþ8ۀ‰€‚€ ‚€€€€€ç抏H€†"€‰€‚ÿ Next: Topological Sorting Up: Applications of Graph Traversal Previous: Tree and Cycle DetectionTwo-Coloring GraphsIn vertex coloring, we seek to assign a color to each vertex of a graph G such that no edge links two vertices of the same color. We can avoid all conflicts by assigning each vertex its own color. However, the goal is to use as few colors as possible. Vertex coloring problems arise often in scheduling applications, such as register allocation in compilers. See Section for a full treatment of vertex coloring algorithms and applications.  %ú«ˆvŽ+ $€õ€€€€‚ÿA graph is bipartite if it can be colored without conflicts while using only two colors. Bipartite graphs are important because they arise often in practice and have more structure than arbitrary graphs. For example, consider the ``had-sex-with'' graph in a heterosexual world. Men have sex only with women, and vice versa. Thus gender defines a legal two-coloring. Irrespective of the accuracy of the model, it should be clear that bipartite graphs are simpler to work with than general graphs.     ]3QŒßÁ* "€g€€‚€‚ÿBut how can we find an appropriate two-coloring of a graph, thus separating the men from the women? Suppose we assume that the starting vertex is male. All vertices adjacent to this man must be female, assuming the graph is indeed bipartite.We can augment either breadth-first or depth-first search so that whenever we discover a new vertex, we colvŽßÁ-ˆor it the opposite of its parent. For each non-discovery edge, we check whether it links two vertices of the same color. Such a conflict means that the graph cannot be two-colored. However, we will have constructed a proper two-coloring whenever we terminate without conflict. We can assign the first vertex in any connected component to be whatever color/sex we wish. Although we can separate the men from the women, we can't tell them apart just using the graph.)vŽÂ& €€€‚ÿÏÎßÁ×à Ё­€€‚ç9ۀ†"€ ‰€çü8ۀ†"€ ‰€çþ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€梞Û/€†"€‰€棞Û/€†"€‰€怞Û/€†"€‰€€ç9ۀ‰€€çü8ۀ‰€€çþ8ۀ‰€€‚‚ÿ Next: Topological Sorting Up: Applications of Graph Traversal Previous: Tree and Cycle Detection AlgorithmsMon Jun 2 23:33:50 EDT 1997)ÂÄ& €€€‚ÿµf×õÄO n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÄæÄ1Uÿÿÿÿÿÿÿÿ—ÿÿÿÿæÄ Å$µÄ Å" €€€ ÿDæÄNÅ1š ÿÿÿÿÿÿÿÿ˜NňÅq Topological Sorting: ňÅ# €.€€‚ÿTopological Sorting íNÅŠÇ øí€€ç,9ۀ†"€ ‰€çü8ۀ†"€ ‰€çÿ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€楞Û/€†"€‰€択Û/€†"€‰€槞Û/€†"€‰€€ç,9ۀ‰€€çü8ۀ‰€€çÿ8ۀ‰€‚€ ‚€†"€]€€‚ÿ Next: Articulation Vertices Up: Applications of Graph Traversal Previous: Two-Coloring GraphsTopological Sorting  Figure: Directed acyclic and cyclic graphs  IìˆÅÓÊ] ˆ€Ý€€€€ç òa €†"€‰€‚€€ç]œ‚€†"€‰€‚ÿA directed, acyclic graph, or DAG, is a directed graph with no directed cycles. Although undirected acyclic graphs are limited to trees, DAGs can be considerably more complicated. They just have to avoid directed cycles, as shown in Figure .   A topological sort of a directed acyclic graph is an ordering on the vertices such that all edges go from left to right. Only an acyclic graph can have a topological sort, because a directed cycle must eventually return home to the source of the cycle. However, every DAG has at least one topological sort, and we can use depth-first search to find such an ordering. Topological sorting proves very useful in scheduling jobs in their proper sequence, as discussed in catalog Section .üŠÇÏÌ: B€‡€€€€†"€^€€‚ÿDepth-first search can be used to test whether a graph is a DAG, and if so to find a topological sort for it. A directed graph is a DAG if and only if no back edges are encountered during a depth-first search. Labeling each of the vertices in the reverse order that they are marked completely-explored finds a topological sort of a DAG. Why? Consider what happens to each directed edge as we encounter it during the exploration of vertex u:—îÓÊfÏ© ß€Ȁ€€€€€€€€€€€€€€€€€€€€‚€€€€€€€€†"€^‚€€€€€€€€€€€€‚ÿu If v is currently undiscovered, then we then start a DFS of v before we can continue with u. Thus v is marked completely-explored before u is, and v appears before u in the topological order, as it must.u If v is discovered but not completely-explored, then is a back edge, which is forbidden in a DAG.u If v is completely-explored, then it will have been so labeled before u. Therefore, u appears before v in the topological order, as it must.,ÏÌ’Ï' € €€€‚ÿ)fÏ»Ï& €€€‚ÿÌ˒Ï“  Ё§€€‚ç,9ۀ†"€ ‰€çü8ۀ†"€»Ï“ Å ‰€çÿ8ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÓžÛ/€†"€‰€æÊžÛ/€†"€‰€æËžÛ/€†"€‰€€ç,9ۀ‰€€çü8ۀ‰€€çÿ8ۀ‰€€‚‚ÿ Next: Articulation Vertices Up: Applications of Graph Traversal Previous: Two-Coloring Graphs AlgorithmsMon Jun 2 23:33:50 EDT 1997)»ÏŒ & €€€‚ÿµf“ q O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Œ ¢ 1Uÿÿÿÿÿÿÿÿ™ÿÿÿÿ¢ Æ $q Æ " €€€ ÿF¢  1 ÿÿÿÿÿÿÿÿš  H - Articulation Vertices<Æ H # €2€€‚ÿArticulation Vertices   d  ø!€€ç#9ۀ†"€ ‰€çü8ۀ†"€ ‰€ç9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÌžÛ/€†"€‰€æÍžÛ/€†"€‰€æΞÛ/€†"€‰€€ç#9ۀ‰€€çü8ۀ‰€€ç9ۀ‰€‚€ ‚€†"€_€€‚ÿ Next: Modeling Graph Problems Up: Applications of Graph Traversal Previous: Topological SortingArticulation Vertices  Figure: An articulation vertex is the weakest point in the graph  ÀH  ] ˆ€…€€ç·j‚b€†"€‰€€€‚€€çكh”€†"€‰€‚ÿSuppose you are a terrorist seeking to disrupt the telephone network. Which station in Figure should you choose to blow up to cause the maximum amount of damage? An articulation vertex is a vertex of a connected graph whose deletion disconnects the graph. Any graph that contains an articulation vertex is inherently fragile, because deleting that single vertex causes a loss of connectivity.   In general, the connectivity of a graph is the smallest number of vertices whose deletion will disconnect the graph. For graphs with an articulation vertex, the connectivity is one. Connectivity is an important measure of robustness in network design, as discussed in catalog Section .   Råd Ó m š€Í€€€€€€€€€€€€€€€€€€çكh”€†"€‰€€‚ÿA simple application of either depth-first or breadth-first search suffices to find all the articulation vertices in a graph in O(n (m+n)). For each vertex v, delete it and then do a BFS traversal of the remaining graph to establish whether it is still connected. In fact, there is a clever O(n+m) algorithm that tests all the vertices using only a single depth-first search. Additional information on edge and vertex connectivity testing appears in Section .) ü & €€€‚ÿS+Ó O ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)ü x & €€€‚ÿµfO - O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1x ^ 1Uÿÿÿÿÿÿÿÿ›ÿÿÿÿ^ ‚ $- ‚ " €€€ ÿH^ Ê 1ÖÿÿÿÿÿÿÿÿœÊ  „Š Modeling Graph Problems>‚  # €6€€‚ÿModeling Graph Problems @Ê T@ / ,‚7€€ç$9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç,9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÏžÛ/€†"€‰€æОÛ/€†"€‰€æÑžÛ/€†"€‰€€ç$9ۀ‰€€çö7ۀ‰€€ç,9ۀ‰€‚€‚€çP¶[\€†"€‰€ç?Ša¯€†"€‰€‚ÿ Next: Minimum Spanning Trees Up: Graph Algorithms Previous: Articulation VerticesModeling Graph ProblemsProper modeling is the key to making effective use of graph algorithms. We have seen a variety of definitions of graph properties, and algorithms for computing them. All told, about two dozen different graph problems are presented in the catalog, mostly in Sections and  T@ ‚ . These problems provide a framework for modeling most applications.  Ž B % €€€‚ÿThe applications below demonstrate the power of proper modeling. Each of them arose in a real-world application as stated, and each can be modeled as a graph problem. Some of the modelings are quite clever, but they illustrate the versatility of graphs in representing relationships. As you read the problem, try to devise an appropriate graph representation before peeking to see how we did it.̝T@ ÔB / ,€;€Ȁ€€€‚ÿu ``I'm looking for an algorithm to design natural routes for video-game characters to follow through an obstacle-filled room. How should I do it?''  íB çC & €Û€Ȁ‚ÿPresumably the route that is wanted is the path that looks most like the one that an intelligent being would choose. Since intelligent beings are either lazy or efficient, this should be modeled as some kind of shortest path problem.`%ÔB GF ; D€M€ȀçUJN€†"€‰€‚ÿBut what is the graph? One approach would be to lay a grid of points in the room and have a vertex for each point that is a valid place for the character to stand, i.e. so it does not lie within an obstacle. There will be an edge between any pair of nearby vertices, weighted according to the distance between them. The shortest path between two vertices will be close to the shortest path between the points. Although direct geometric methods are known for shortest paths (see Section ), it is easier to model this discretely as a graph.  Ë çC H + $€A€Ȁ€€‚ÿu ``In DNA sequencing, we are given experimental data consisting of small fragments. For each fragment f, we have certain other fragments that are forced to lie to the left of f, certain fragments forced to be to the right of f, and the remaining fragments, which are free to go on either side. How can we find a consistent ordering of the fragments from left to right that satisfies all the constraints?''     RüGF dJ V z€ý€Ȁ†"€`€€€€†"€a€€€€€€‚ÿCreate a directed graph, where each fragment is assigned a unique vertex. Insert a directed edge from any fragment l that is forced to be to the left of f, and a directed edge to any fragment r forced to be to the right of f. We seek an ordering of the vertices such that all the edges go from left to right. This is exactly a topological sort of the resulting directed acyclic graph. The graph must be acyclic for this to work, because cycles make finding a consistent ordering impossible.žsH L + $€ç€Ȁ€€‚ÿu ``In my graphics work I need to solve the following problem. Given an arbitrary set of rectangles in the plane, how can I distribute them into a minimum number of buckets such that the subset of rectangles in the same bucket do not intersect each other? In other words, there should not be any overlapping area between any two rectangles in the same bucket.''     þÂdJ O < F€…€Ȁ€€€€‚€€€‚ÿWe can formulate a graph where each vertex is a rectangle, and there is an edge if two rectangles intersect. Each bucket corresponds to an independent set of rectangles, so there is no overlap between any two. A vertex coloring of a graph is a partition of the vertices into independent sets, so minimizing the number of colors is exactly what you want.u ``In porting code from UNIX to DOS, I have to shorten the names of several hundred files down to at most 8 characters each. I can't just take the first eight characters from each name, because ``filename1'' and ``filename2'' will get assigned the exact same name. How can I shorten the names while ensuring that they do not collide?''     Y÷L e b ’€÷€Ȁ†"€b†"€ç†"€c€€€€çg[ր†"€‰€‚ÿConstruct a graph with vertices corresponding to each original file name for , as well as a collection of acceptable shortenings for each name . Add aO e ‚ n edge between each original and shortened name. Given such a formulation, we seek a set of n edges that have no vertices in common, because the file name of each is thus mapped to a distinct acceptable substitute. Bipartite matching, discussed in Section , is exactly this problem of finding an independent set of edges in a graph.  .O “‚ + $€€Ȁ€€‚ÿu ``In organized tax fraud, criminals submit groups of phony tax returns in the hopes of getting undeserved refunds. These phony returns are all similar, but not identical. How can we detect clusters of similar forms so the IRS can nail the cheaters?''  É„e \… E X€ €Ȁ€€€€†"€d‚€€€‚ÿA natural graph model treats each form as a vertex and adds an edge between any two tax forms that are suspiciously similar. A cluster would correspond to a group of forms with many edges between them. In particular, a clique is a set of k vertices with all possible edges between them. Any sufficiently large clique identifies a cluster worth studying.   u ``In the optical character-recognition system that we are building, we need a way to separate the lines of text. Although there is some white space between the lines, problems like noise and the tilt of the page makes it hard to find. How can we do line segmentation?     5 “‚ ‘‡ , &€€Ȁ€€‚ÿConsider the following graph formulation. Treat each pixel in the image as a vertex in the graph, with an edge between two neighboring pixels. The weight of this edge should be proportional to how dark the pixels are. A segmentation between two lines is a path in this graph from the left to right side of the page. Of all possible paths, we seek a relatively straight path that avoids as much blackness as possible. This suggests that the shortest path in the pixel graph will likely find a good line segmentation.,\… œ‡ ' € €€€‚ÿ)‘‡ æ‡ & €€€‚ÿÀ¿œ‡ Š‰  Ё€€‚ç$9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç,9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÒžÛ/€†"€‰€æ7ÀÛ/€†"€‰€æ.ÀÛ/€†"€‰€€ç$9ۀ‰€€çö7ۀ‰€€ç,9ۀ‰€€‚‚ÿ Next: Minimum Spanning Trees Up: Graph Algorithms Previous: Articulation Vertices AlgorithmsMon Jun 2 23:33:50 EDT 1997)æ‡ Ï‰ & €€€‚ÿµfŠ‰ „Š O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ω µŠ 1UÿÿÿÿÿÿÿÿÿÿÿÿµŠ ي $„Š ي " €€€ ÿGµŠ ‹ 1ß ÿÿÿÿÿÿÿÿž ‹ ]‹  Å Minimum Spanning Trees=ي ]‹ # €4€€‚ÿMinimum Spanning Trees Ы ‹ -Ž % ‚g€€ç%9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç#9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ/ÀÛ/€†"€‰€æ0ÀÛ/€†"€‰€æ1ÀÛ/€†"€‰€€ç%9ۀ‰€€çö7ۀ‰€€ç#9ۀ‰€‚€‚€‚€€€€€€€€€€‚ÿ Next: Prim's Algorithm Up: Graph Algorithms Previous: Modeling Graph ProblemsMinimum Spanning Trees A tree is a connected graph with no cycles. A spanning tree is a subgraph of G that has the same set of vertices of G and is a tree. A minimum spanning tree of a weighted graph G is the spanning tree of G whose edges sum to minimum weight.  zõ]‹ ³À … ؀ù€€†"€e€€‚ç±÷Mj€†"€‰€†"€f†"€g†"€&†"€*çô»G¹€†"€‰€‚ÿ  Figure: Two spanning trees of point set (a); the minimum spanning tree (b), and the shortest path from center tree (c)  Minimum spanning trees are useful in finding the least amount of wire necessary to connect a group of homes or cities, as illustrated in Figure . In such geometric problems, the point set defines -Ž ³À ي a complete graph, with edge assigned a weight equal to the distance from to . Additional applications of minimum spanning trees are discussed in Section .©i-Ž \à @ N€Ó€€€€€€€€€€€‚ÿA minimum spanning tree minimizes the total length over all possible spanning trees. However, there can be more than one minimum spanning tree in any graph. Consider a graph G with m identically weighted edges. All spanning trees of G are minimum spanning trees, since each contains exactly n-1 equal-weight edges. For general weighted graphs, however, the problem of finding a minimum spanning tree is more difficult. It can, however, be solved optimally using two different greedy algorithms. Both are presented below, to illustrate how we can demonstrate the optimality of certain greedy heuristics.  )³À …à & €€€‚ÿ&\à «Ã # €€€‚ÿu5…Ã Ä @ P€j€Ȁç†Tɀ‰€‚€çy[(€‰€‚ÿu Prim's Algorithm u Kruskal's Algorithm&«Ã FÄ # €€€‚ÿ) Ä oÄ & €€€‚ÿS+FÄ ÂÄ ( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)oÄ ëÄ & €€€‚ÿµfÂÄ  Å O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ëÄ ÑÅ 1UÿÿÿÿÿÿÿÿŸÿÿÿÿÑÅ õÅ $ Å õÅ " €€€ ÿAÑÅ 6Æ 1ÿÿÿÿÿÿÿÿ 6Æ mÆ Ô!Prim's Algorithm7õÅ mÆ # €(€€‚ÿPrim's Algorithm t6Æ îÈ  èù€€ç&9ۀ†"€ ‰€ç$9ۀ†"€ ‰€ç$9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ2ÀÛ/€†"€‰€æ3ÀÛ/€†"€‰€æ4ÀÛ/€†"€‰€€ç&9ۀ‰€€ç$9ۀ‰€€ç$9ۀ‰€‚€ ‚€‚€€‚ÿ Next: Kruskal's Algorithm Up: Minimum Spanning Trees Previous: Minimum Spanning TreesPrim's Algorithm Every vertex will appear in the minimum spanning tree of any connected graph G. Prim's minimum spanning tree algorithm starts from one vertex and grows the rest of the tree one edge at a time.Ó“mÆ ÁÊ @ N€+€€‚†"€h€€†"€i‚ÿIn greedy algorithms, we make the decision of what to do next by selecting the best local option from all available choices without regard to the global structure. Since we seek the tree of minimum weight, the natural greedy algorithm for minimum spanning tree repeatedly selects the smallest weight edge that will enlarge the tree.    Figure: Where Prim's algorithm goes bad? No, because  Å|îÈ †Í I `€û€€‚‚‚‚‚†"€j‚€‚ÿ Prim-MST(G) Select an arbitrary vertex to start the tree from. While (there are still non-tree vertices) Select the edge of minimum weight between a tree and non-tree vertex Add the selected edge and vertex to the tree . Prim's algorithm clearly creates a spanning tree, because no cycle can be introduced by adding edges between tree and non-tree vertices. However, why should it be of minimum weight over all spanning trees? We have seen ample evidence of other natural greedy heuristics that do not yield a global optimium. Therefore, we must be particularly careful to demonstrate any such claim.ŸÁÊ /!ß Œ‘€€€€€€€€†"€j†"€k€€€€€€€€€€†"€k†"€l†"€m†"€j†"€n†"€l€€€€€€€€€€€€†"€l†"€k€€€€‚ÿSuppose that there existed a graph G for which Prim's algorithm did not return a minimum spanning tree. Since we are building the tree incrementally, this means that there must have been some particular instant where we went wrong. Before we inserted edge (x,y), consisted of a set of edges that was a subtree of a minimum spanning tree , but choosing edge (x,y) took us away from a minimum span†Í /!õÅ ning tree. But how could it? There must be a path p from x to y in , using an edge , where is in but is not. This edge must have weight at least that of (x,y), or else Prim's algorithm would have selected it instead of (x,y) when it had the chance. Inserting (x,y) and deleting from leaves a spanning tree no larger than before, meaning that Prim's algorithm could not have made a fatal mistake in selecting edge (x,y). Therefore, by contradiction, Prim's algorithm has to construct a minimum spanning tree.Ú‹†Í !O l€€€€€€€€€€€†"€€€€‚ÿPrim's algorithm is correct, but how efficient is it? That depends on which data structures are used to implement it, but it should be clear that O(nm) time suffices. In each of n iterations, we will scan through all the m edges and test whether the current edge joins a tree with a non-tree vertex and whether this is the smallest edge seen thus far. By maintaining a Boolean flag along with each vertex to denote whether it is in the tree or not, this test can be performed in constant time. In fact, better data structures lead to a faster, , implementation by avoiding the need to sweep through more than n edges in any iteration.)/!2!& €€€‚ÿÄà !ö! Ё—€€‚ç&9ۀ†"€ ‰€ç$9ۀ†"€ ‰€ç$9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ5ÀÛ/€†"€‰€æ6ÀÛ/€†"€‰€掟Û/€†"€‰€€ç&9ۀ‰€€ç$9ۀ‰€€ç$9ۀ‰€€‚‚ÿ Next: Kruskal's Algorithm Up: Minimum Spanning Trees Previous: Minimum Spanning Trees AlgorithmsMon Jun 2 23:33:50 EDT 1997)2!!& €€€‚ÿµfö!Ô!O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1!!1Uÿÿÿÿÿÿÿÿ¡ÿÿÿÿ!)!$Ô!)!" €€€ ÿD!m!1°ÿÿÿÿÿÿÿÿ¢m!§!$J!Kruskal's Algorithm:)!§!# €.€€‚ÿKruskal's Algorithm “Œm!: ! ܁)€€ç'9ۀ†"€ ‰€ç$9ۀ†"€ ‰€ç%9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ«ŸÛ/€†"€‰€欟Û/€†"€‰€æ­ŸÛ/€†"€‰€€ç'9ۀ‰€€ç$9ۀ‰€€ç%9ۀ‰€‚€ ‚€‚‚ÿ Next: Shortest Paths Up: Minimum Spanning Trees Previous: Prim's AlgorithmKruskal's Algorithm Kruskal's algorithm is an alternative approach to finding minimum spanning trees that is more efficient on sparse graphs. Like Prim's, Kruskal's algorithm is greedy; unlike Prim's, it does not start with a particular vertex.  `;§!š !% €w€€‚ÿKruskal's algorithm works by building up connected components of the vertices. Initially, each vertex forms its own separate component in the tree-to-be. The algorithm repeatedly considers the lightest remaining edge and tests whether the two endpoints lie within the same connected component. If so, the edge will be discarded, because adding it will create a cycle in the tree-to-be. If the endpoints are in different components, we insert the edge and merge the components. Since each connected component is always a tree, we need never explicitly test for cycles:€t: !JC!0 .‚ù€€‚€€‚‚€€‚€€€€‚€€€€‚€€†"€o€€‚†"€p‚€€€€‚€€€€€€€€€†"€p†"€k€€€€†"€k€€€€€€€€€€€€†"€l€€€€†"€q†"€k€€€€‚ÿ Kruskal-MST(G) Put the edges in a priority queue ordered by weight. count=0 while (count < n-1) do get next edge (v,w) if (component (v) component(w)) add to merge component(v) and component(w) This algorithm adds n-1 edges without crš !JC!)!eating a cycle, so clearly it creates a spanning tree of any connected graph. But why must this be a minimum spanning tree? Suppose it wasn't. As with the correctness proof of Prim's algorithm, there must be some graph for which it fails, and in particular there must a single edge (x,y) whose insertion first prevented the tree from being a minimum spanning tree . Inserting edge (x,y) in will create a cycle with the path from x to y. Since x and y were in different components at the time of inserting (x,y), at least one edge on this path would have been considered by Kruskal's algorithm after (x,y) was. But this means that , so exchanging the two edges yields a tree of weight at most . Therefore, we could not have made a mistake in selecting (x,y), and the correctness follows.²š !NE!R r€g€€€€†"€r€€€€€€€€€€‚ÿWhat is the time complexity of Kruskal's algorithm? Inserting and retrieving m edges from a priority queue such as a heap takes time. The while loop makes at most m iterations, each testing the connectivity of two trees plus an edge. In the most simple-minded approach, this can be implemented by a breadth-first or depth-first search in a graph with at most n edges and n vertices, thus yielding an O(mn) algorithm.»JC!dG![ „€}€€€€€€çÕ•€†"€‰€†"€W†"€r€‚ÿHowever, a faster implementation would result if we could implement the component test in faster than O(n) time. In fact, the union-find data structure, discussed in Section , can support such queries in time. With this data structure, Kruskal's algorithm runs in time, which is faster than Prim's for sparse graphs. Observe again the impact that the right data structure can have in implementing a straightforward algorithm.)NE!G!& €€€‚ÿ¹ždG!FI! Ё€€‚ç'9ۀ†"€ ‰€ç$9ۀ†"€ ‰€ç%9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€殟Û/€†"€‰€毟Û/€†"€‰€æ°ŸÛ/€†"€‰€€ç'9ۀ‰€€ç$9ۀ‰€€ç%9ۀ‰€€‚‚ÿ Next: Shortest Paths Up: Minimum Spanning Trees Previous: Prim's Algorithm AlgorithmsMon Jun 2 23:33:50 EDT 1997)G!oI!& €€€‚ÿµfFI!$J!O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1oI!UJ!1Uÿÿÿÿÿÿÿÿ£ÿÿÿÿUJ!yJ!$$J!yJ!" €€€ ÿ?UJ!žJ!1Žÿÿÿÿÿÿÿÿ€žJ!íJ!Qƒ!Shortest Paths5yJ!íJ!# €$€€‚ÿShortest Paths žJ!M! ‚€€ç(9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç&9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€江Û/€†"€‰€沟Û/€†"€‰€泟Û/€†"€‰€€ç(9ۀ‰€€çö7ۀ‰€€ç&9ۀ‰€‚€‚€†"€s€€€€‚ÿ Next: Dijkstra's Algorithm Up: Graph Algorithms Previous: Kruskal's AlgorithmShortest Paths  Figure: The shortest path from to t can pass through many intermediate vertices.  ÈíJ!"€!F Z€“€€€€€€çÒ¢a‡€†"€‰€‚ÿThe shortest path between two vertices and t in an unweighted graph can be constructed using a breadth-first search from . When we first encounter t in the search, we will have reached it from using the minimum number of possible edges. This minimum-link path is recorded in the breadth-first search tree, and it provides the shortest path when all edges have equal weight. However, in an arbitrary weighted graph, the weight of a path between two vertices is the sum of the weights of the edges along the path. The shortest path might use a large number of edges, just as the shortest route (timewise) from home to office may involve shortcuts using backroads and many turns, as showM!"€!yJ!n in Figure .  ã‘M!!R r€'€€çUJN€†"€‰€çUÜ{€†"€‰€€‚ÿShortest paths have a surprising variety of applications. See catalog Section and the war story of Section for further examples.)"€!.!& €€€‚ÿ&!T!# €€€‚ÿ}=.!с!@ P€z€Ȁç;1h怉€‚€眿nE€‰€‚ÿu Dijkstra's Algorithm u All-Pairs Shortest Path&T!÷!# €€€‚ÿ)с! ‚!& €€€‚ÿS+÷!s‚!( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997) ‚!œ‚!& €€€‚ÿµfs‚!Qƒ!O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1œ‚!‚ƒ!1Uÿÿÿÿÿÿÿÿ¥ÿÿÿÿ‚ƒ!Šƒ!$Qƒ!Šƒ!" €€€ ÿE‚ƒ!ëƒ!12ÿÿÿÿÿÿÿÿŠëƒ!&„!—Ã!Dijkstra's Algorithm;Šƒ!&„!# €0€€‚ÿDijkstra's Algorithm {nëƒ!¡‡!  èí€€ç)9ۀ†"€ ‰€ç'9ۀ†"€ ‰€ç'9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æߟÛ/€†"€‰€æÖŸÛ/€†"€‰€æןÛ/€†"€‰€€ç)9ۀ‰€€ç'9ۀ‰€€ç'9ۀ‰€‚€ ‚€‚€€‚ÿ Next: All-Pairs Shortest Path Up: Shortest Paths Previous: Shortest PathsDijkstra's Algorithm We can use Dijkstra's algorithm to find the shortest path between any two vertices (,t) in a weighted graph, where each edge has non-negative edge weight. Although most applications of shortest path involve graphs with positive edge weights, such a condition is not needed for either Prim's or Kruskal's algorithm to work correctly. The problems that negative edges cause Dijkstra's algorithm will become apparent once you understand the algorithm.  P¶&„!ñ‰!š w€€†"€t€€€€†"€€€†"€u€€†"€v†"€u€€€€€€€€€€€€€€€€‚ÿThe principle behind Dijkstra's algorithm is that given the shortest path between and each of a given set of vertices , there must exist some other vertex x such that the shortest path from to x must go from to to x, for some . Specifically, it is the vertex x that minimizes over all , where w(i,j) is the length of the edge from i to j and dist(i,j) is the length of the shortest path between them.-Ò¡‡!Œ![ „€¥€€€€€€€€€€€€€€€€€€€€‚ÿThis suggests a dynamic programming-like strategy. The shortest path from to itself is trivial unless there are negative weight edges, so dist(,)=0. Armed with the shortest path to , if (,y) is the lightest edge incident to , then d(,y) = w(,y). As soon as we decide that we have determined the shortest path to a node x, we search through all the outgoing edges of x to see whether there is a better path from to some unknown vertex through x:Úñ‰!ø!Ø ~€€‚€€‚†"€w‚€€€€†"€x‚€€€€€€€€€€‚€€‚†"€y‚†"€z€€€€‚†"€{†"€|‚†"€}‚†"€~‚ÿ ShortestPath-Dijkstra(G,s,t) for i=1 to n, for each edge (, v), dist[v]=w(, v) last= while ( ) select , the unknown vertex minimizing dist[v] for each edge , Ô•Œ!ØÀ!? L€-€€€€‚†"€€€€‚ÿTo be certain of finding the shortest path between and t, we might have to first find the shortest path between and all other vertices. This defines a shortest path spanning tree rooted in . For undirected graphs, this will be the breadth-first search tree, but in general it provides the shortest path between and all other vertices.What is the running time of this algorithm? When implemented using adjacency lists and a Boolean array to mark what iø!ØÀ!Šƒ!s known about each vertex, the complexity is . This is the same running time as a proper version of Prim's algorithm; indeed, except for the extension condition, it is the same algorithm as Prim's.)ø!Á!& €€€‚ÿž·ØÀ!¹Â! Ё€€‚ç)9ۀ†"€ ‰€ç'9ۀ†"€ ‰€ç'9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æØŸÛ/€†"€‰€æÙŸÛ/€†"€‰€æÚŸÛ/€†"€‰€€ç)9ۀ‰€€ç'9ۀ‰€€ç'9ۀ‰€€‚‚ÿ Next: All-Pairs Shortest Path Up: Shortest Paths Previous: Shortest Paths AlgorithmsMon Jun 2 23:33:50 EDT 1997)Á!âÂ!& €€€‚ÿµf¹Â!—Ã!O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1âÂ!ÈÃ!1Uÿÿÿÿÿÿÿÿ§ÿÿÿÿÈÃ!ìÃ!$—Ã!ìÃ!" €€€ ÿHÈÃ!4Ä!1G ÿÿÿÿÿÿÿÿš4Ä!rÄ!ˆÏ!All-Pairs Shortest Path>ìÃ!rÄ!# €6€€‚ÿAll-Pairs Shortest Path Ƨ4Ä!8Ç! ‚c€€ç*9ۀ†"€ ‰€ç'9ۀ†"€ ‰€ç(9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÛŸÛ/€†"€‰€æÜŸÛ/€†"€‰€æÝŸÛ/€†"€‰€€ç*9ۀ‰€€ç'9ۀ‰€€ç(9ۀ‰€‚€ ‚€‚†"€€€†"€‚ÿ Next: War Story: Nothing but Up: Shortest Paths Previous: Dijkstra's AlgorithmAll-Pairs Shortest Path If we want to find the length of the shortest path between all pairs of vertices, we could run Dijkstra's algorithm n times, once from each possible starting vertex. This yields a cubic time algorithm for all-pairs shortest path, since .  Œ5rÄ!ÄÉ!W |€o€€†"€Q‚€€€€†"€€€€€€€€‚ÿCan we do better? Significantly improving the complexity is still an open question, but there is a superslick dynamic programming algorithm that also runs in .  There are several ways to characterize the shortest path between two nodes in a graph. The Floyd-Warshall algorithm starts by numbering the vertices of the graph from 1 to n. We use these numbers here not to label the vertices, but to order them. Define to be the length of the shortest path from i to j using only vertices numbered from 1, 2,..., k as possible intermediate vertices.Û”8Ç!ŸË!G \€-€€€€†"€€€‚†"€‚‚‚ÿWhat does this mean? When k = 0, we are allowed no intermediate vertices, so that every path consists of at most one edge. Thus . In general, adding a new vertex k+1 as a possible intermediary helps only if there is a short path that goes through it, soThis recurrence performs only a constant amount of work per cell. The following dynamic programming algorithm implements the recurrence:‡ÄÉ!ŠÌ!€ ΀€€‚€€‚†"€ƒ€€‚€€€€‚€€€€‚€€€€‚†"€„‚ÿ Floyd(G) Let , the weight matrix of G for k=1 to n for i=1 to n for j=1 to n ˆQŸË!.Î!7 <€¥€€†"€Q€€€‚ÿThe Floyd-Warshall all-pairs shortest path runs in time, which asymptotically is no better than n calls to Dijkstra's algorithm. However, the loops are so tight and the program so short that it runs better in practice. It is also notable as one of the rare algorithms that work better on adjacency matrices than adjacency lists.)ŠÌ!WÎ!& €€€‚ÿS+.Î!ªÎ!( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)WÎ!ÓÎ!& €€€‚ÿµfªÎ!ˆÏ!O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÓÎ!¹Ï!1Uÿÿÿÿÿÿÿÿ©ÿÿÿÿ¹Ï! "$ˆÏ! "" €€€ ÿ¹Ï! "ˆÏ!L¹Ï!X"1.ÿÿÿÿÿÿÿÿªX"š"~Š"War Story: Nothing but NetsB "š"# €>€€‚ÿWar Story: Nothing but Nets ÿX"ž" ؁€€ç+9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç)9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÞŸÛ/€†"€‰€æ ¿Û/€†"€‰€æ¿Û/€†"€‰€€ç+9ۀ‰€€çö7ۀ‰€€ç)9ۀ‰€‚€‚€‚ÿ Next: War Story: Dialing for Up: Graph Algorithms Previous: All-Pairs Shortest PathWar Story: Nothing but NetsI'd been tipped off about a small printed-circuit board testing company nearby that was in need of some algorithmic consulting. And so I found myself inside a typically non-descript building in a typically non-descript industrial park, talking with the president of Integri-Test, along with one of his lead technical people.   +öš"É"5 8€í€€€€€€‚‚‚ÿ``We're the leaders in robotic printed-circuit board testing devices. Our customers have very high reliability requirements for their PC-boards. They must check that each and every board has no wire breaks before filling it with components. This means testing that each and every pair of points on the board that are supposed to be connected are connected.''``How do you do the testing?'' I asked.``We have a robot with two arms, each with electric probes. To test whether two points are properly connected, the arms simultaneously contact both of the points. If the two points are propertly connected, then the probes will complete a circuit. For each net, we hold one arm fixed at one point and move the other to cover the rest of the points.''7þž" "9 @€ÿ€€‚†"€…€€‚‚ÿ``Wait!'' I cried. ``What is a net?''  Figure: An sample net showing (a) the metal connection layer, (b) the contact points, (c) their minimum spanning tree, and (d) the points partitioned into clusters  ``On a circuit board there are certain sets of points that are all connected together with a metal layer. This is what we mean by a net. Sometimes a net consists of two points, i.e. is an isolated wire. Sometimes a net can have 100 to 200 points, like all the connections to power or ground.''ÓÉ" "< F€©€€‚ç÷Mu€€†"€‰€‚ÿ``I see. So you have a list of all the connections between pairs of points on the circuit board, and you want to trace out these wires.''He shook his head. ``Not quite. The input for our testing program consists only of the net contact points, as shown in Figure (b). We don't know where the actual wires are, but we don't have to. All we have to do is verify that all the points in a net are connected together. We do this by putting the left robot arm on the leftmost point in the net, then having the right arm move around to all the other points in the net to test if they are connected to the left point. If they are all connected to the left point, it means that they must all be connected to each other.''Å ""@"B R€€€‚‚‚‚çÃß{+€†"€‰€‚ÿI thought for a moment about what this meant. ``OK. So your right arm has to visit all the other points in the net. How do you choose the order to visit them?''The technical guy spoke up. ``Well, we sort the points from left to right and then go in that order. Is that a good thing to do?''``Have you ever heard of the traveling salesman problem?'' I asked.  He was an electrical engineer, not a computer scientist. ``No, what's that?'' he asked.``Traveling salesman is the name of the exact problem that you are trying to solve. Given a set of points you have to visit, how do you order them so as to minimize the travel time. Algorithms for the traveling salesman problem have been extensively studied. For small nets, by doing an exhaustive search you will be able to find the optimal tour. For big nets, there are heuristics that will get you very close to the optimal tour.'' I would have pointed them to Section if I had  ""@" "had this book handy. ã ",D"' €Ç€€‚‚ÿThe president scribbled down some notes and then frowned. ``Fine. Maybe you can order the points in a net better for us. But that is not our real problem. When you watch our robot in action, the right arm sometimes has to run all the way to the right side of the board on a given net, while the left arm just sits there. It seems we would benefit by breaking a net into smaller pieces to balance things out.''I sat down and thought. The left and right arms were each going to have interlocking TSP problems to solve. The left arm would move between the leftmost points of each net, while the right arm was going to visit all the other points in each net as ordered by the left TSP tour. By breaking each net into smaller nets, so that each net occupies a small chunk of real estate, we would avoid making the right arm cross all the way across the board. Further, a lot of little nets meant there would be more points in the left TSP, so each left-arm movement was likely to be short, too.ð"@"AF"% €á€€‚ÿ``You are right. We should win if we can break big nets into small nets. We want the nets to be small, both in the number of points and in the area of the net. But we must be sure that if we validate the connectivity of each small net, we will have confirmed that the big net is connected. Whenever there is one point in common between two little nets, that is enough to show that the bigger net formed by the two little nets is connected, since current can flow between any pair of points.''Òƒ,D"J"O l€ €€çô»G¹€†"€‰€ç÷Mu€€†"€‰€‚ÿNow we had to break each net into overlapping pieces, where each piece was small. This is a clustering problem. Minimum spanning trees are often used for clustering, as discussed in Section . In fact, that was the answer! We could find the minimum spanning tree of the net points and break it into little clusters whenever a spanning tree edge got too long. As shown in Figure (d), each cluster would share exactly one point in common with another cluster, with connectivity ensured because we are covering the edges of a spanning tree. The shape of the clusters would reflect the points in the net, exactly as we would want. If the points lay along a line across the board, the minimum spanning tree would be a path, and the clusters would be pairs of points. If the points all fell in a tight region, there would be one nice fat cluster that the right arm would just scoot around.   ìAF"(L") €Ù€€‚‚‚ÿSo I explained the idea of constructing the minimum spanning tree of a graph. The boss listened, scribbled more notes, and frowned again.``I like your clustering idea. But these minimum spanning trees you talk about are defined on graphs. All you got are points. Where do the weights of the edges come from?''``Oh, we can think of it as a complete graph, where every pair of points are connected. The weight of the edge defined by the two points is simply the distance. Or is it...?''pGJ"˜N") €€€‚‚‚ÿI went back to thinking. The edge cost between two points should reflect the travel time between them. While the distance was related to the travel time, it wasn't necessarily exactly the same thing.``Hey. I have a question about your robot. Does it take the same amount of time to move the arm left-right as it does up-down?''They thought a minute. ``Yeah, it does. We use the same type of motors to control horizontal and vertical movements. Further, since the two motors for each arm are independent, we can simultaneously move each arm both horizontally and vertically.'' Ð(L"®€": B€£€€€€€€†"€†‚ÿ``That so? The time to move both one foot left and one foot up is exactly the same as just moving one foot left? This means that the weight cost for each edge in the graph should not be the Euclidean distance between the two points, but the biggest difference between either the x- or y-coordinat˜N"®€" "e. This is something we call the metric, but we can capture it by changing the edge weights in the graph. Anything else funny about your robots?'' I asked.   »’˜N"i„") €%€€‚‚‚ÿ``Well, it takes some time for the robot to come up to speed. We should probably also factor in acceleration and deceleration of the arms.''``Darn right. The more accurately you can model the time your arm takes to move between two points, the better our solution will be. But now we have a very clean formulation. Let's code it up and let's see how well it works!''They were somewhat skeptical whether this approach would do any good, but they agreed to think about it. A few weeks later they called me back and reported that the new algorithm reduced testing time by about 30% over their previous approach, at a cost of a little more computational preprocessing. However, since their testing machine costs $200,000 a pop and a PC costs $2,000, this is an excellent tradeoff. It is particularly advantageous since the preprocessing need only be done once when testing multiple instances of the same board.L$®€"µ‡"( €I€€€‚ÿThe key idea leading to the successful solution was knowing how to model the job in terms of classical algorithmic graph problems. I smelled TSP the instant they started talking about minimizing robot motion. Once I realized that they were implicitly forming a star-shaped spanning tree to ensure connectivity, it was natural to ask whether a minimum spanning tree would perform any better. This idea led to a natural way to think about clustering, and thus partitioning each net into smaller nets. Finally, by carefully constructing our distance metric to accurately model the costs of the robot itself, we get to incorporate quite complicated properties (such as acceleration and differences between horizontal and vertical speeds) without changing our fundamental graph model or algorithm design.)i„"އ"& €€€‚ÿÂÁµ‡" ‰" Ё“€€‚ç+9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç)9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ¿Û/€†"€‰€æ¿Û/€†"€‰€æ¿Û/€†"€‰€€ç+9ۀ‰€€çö7ۀ‰€€ç)9ۀ‰€€‚‚ÿ Next: War Story: Dialing for Up: Graph Algorithms Previous: All-Pairs Shortest Path AlgorithmsMon Jun 2 23:33:50 EDT 1997)އ"ɉ"& €€€‚ÿµf ‰"~Š"O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ɉ"¯Š"1Uÿÿÿÿÿÿÿÿ«ÿÿÿÿ¯Š"ӊ"$~Š"ӊ"" €€€ ÿQ ¯Š"$‹"1:ÿÿÿÿÿÿÿÿ¬$‹"k‹"‚#War Story: Dialing for DocumentsG$ӊ"k‹"# €H€€‚ÿWar Story: Dialing for Documents ©œ$‹""  èI€€çW9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç*9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ¿Û/€†"€‰€æ¿Û/€†"€‰€æ¿Û/€†"€‰€€çW9ۀ‰€€çö7ۀ‰€€ç*9ۀ‰€‚€‚€‚€€‚ÿ Next: Exercises Up: Graph Algorithms Previous: War Story: Nothing butWar Story: Dialing for Documents I was part of a group visiting Periphonics, an industry leader in building telephone voice-response systems. These are more advanced versions of the Press 1 for more options, Press 2 if you didn't press 1 telephone systems that have come to blight everyone's lives in recent years. We were being given the standard tour when someone from our group asked, ``Why don't you guys use voice recognition for data entry. It would be a lot less annoying than typing things out on the keypad.''   k>k‹"‹Â"- (€}€€‚€€‚ÿThe tour guide reacted smoothly. ``Our customers have the option of incorporating speech recognition into our products, but very few of them do. User-independent, connected-speech recogniti"‹Â"ӊ"on is not accurate for most applications. Our customers prefer building systems around typing text on the telephone keyboards.''``Prefer typing, my pupik!'', came a voice from the rear of our group. ``I hate typing on a telephone. Whenever I call my brokerage house to get stock quotes, I end up talking to some machine, which asks me to type in the three letter code. To make it worse, I have to hit two buttons to type in one letter, in order to distinguish between the three letters printed on each key of the telephone. I hit the 2 key and it says Press 1 for `A', Press 2 for `B', Press 3 for `C'. Pain the neck if you ask me.''†["Å"+ $€·€€‚‚‚‚ÿ``Maybe you don't really have to hit two keys for each letter?'' I chimed in. ``Maybe the system could figure out the correct letter from context?''``There isn't a whole lot of context when you type in three letters of stock market code.''``Sure, but there would be plenty of context if we were typing in English sentences. I'll bet that we could reconstruct English sentences correctly if they were typed in a telephone at one keystroke per letter.''The guy from Periphonics gave me a disinterested look, then continued the tour. But when I got back to the office, I decided to give it a try.ᶋÂ"òÇ"+ $€m€€€€‚ÿIt was clear that not all letters were equally likely to be typed on a telephone. In fact, not all letters can be typed, since `Q' and `Z' are not labeled on the standard American telephone. Therefore, we adopted the convention that `Q', `Z', and space were all on the * key. We could take advantage of the uneven distribution of letter frequencies to help us decode the text. For example, if you hit the 3 key while typing English, you were more likely to have meant to type an `E' than either a `D' or `F'. By taking into account the frequencies of a window of three characters, we could predict the typed text. Indeed, this is what happened when I tried it on the Gettysburg Address:  Õ¯Å"ÇÈ"& €_€Ȁ‚ÿenurraore ane reten yeasr ain our ectherr arotght eosti on ugis aootinent a oey oation aoncdivee in licesty ane eedicatee un uhe rrorosition uiat all oen are arectee e ual›uòÇ"bÊ"& €ë€Ȁ‚ÿony ye are enichde in a irect aitil yar uestini yhethes uiat oatioo or aoy oation ro aoncdivee ane ro eedicatee aan loni eneure ye are oet on a irect aattlediele oe uiat yar ye iate aone un eedicate a rostion oe uiat eiele ar a einal restini rlace eor uiore yin iere iate uhdis lives uiat uhe oation ogght live it is aluniethes eittini ane rrores uiat ye rioule en ugirùÓÇÈ"[Í"& €§€Ȁ‚ÿatt in a laries reore ye aan oou eedicate ye aan oou aoorearate ye aan oou ialloy ugis iroune the arate oen litini ane eeae yin rustgilee iere iate aoorearatee it ear aante our roor rowes un ade or eeuraat the yople yill little oote oor loni renences yiat ye ray iere att it aan oetes eosiet yiat uhfy eie iere it is eor ur uhe litini rathes un ae eedicatee iere un uhe undiniside yopl yhici uhfy yin entght iere iate uiur ear ro onaky aetancde it is rathes eor ur un ae iere eedicatee un uhe irect uarl rencinini adeore ur uiat eron uhere ioooree eeae ye uale inarearee eeuotion uo tiat aaure eor yhici uhfy iere iate uhe lart eull oearure oe eeuotioo tiat ye iere iggily rerolue uiat uhere eeae riall oou iate eide ioüËbÊ"c#1 0€—€€€€€€‚ÿThe trigram statistics did a decent job of translating it into Greek, but a terrible job of transcribing English. One reason was clear. This algorithm knew nothing about English words. If we coupled it with a dictionary, we might be on to something. The difficulty was that often two words in the dictionary would be represented by the exact same string of phone codes. For an extreme example, the code string ``22737'' collides with eleven distinct English words, including cases, cares, cards, capes, caper, and bases. As a first attempt, we reported the unambiguous characters of any words that collided in the dictiona[Í"c#ӊ"ry, and used trigrams to fill in the rest of the characters. We were rewarded with:  Õ¯[Í"8#& €_€Ȁ‚ÿeourscore and seven yearr ain our eatherr brought forth on this continent azoey nation conceivee in liberty and dedicatee uo uhe proposition that all men are createe equal›uc#Ó#& €ë€Ȁ‚ÿony ye are engagee in azipeat civil yar uestioi whether that nation or aoy nation ro conceivee and ro dedicatee aan long endure ye are oet on azipeat battlefield oe that yar ye iate aone uo dedicate a rostion oe that field ar a final perthni place for those yin here iate their lives that uhe nation oight live it is altogether fittinizane proper that ye should en thisšt8#m#& €é€Ȁ‚ÿaut in a larges sense ye aan oou dedicate ye aan oou consecrate ye aan oou hallow this ground the arate men litioi and deae yin strugglee here iate consecratee it ear above our roor power uo ade or detract the world will little oote oor long remember what ye ray here aut it aan meter forget what uhfy die here it is for ur uhe litioi rather uo ae dedicatee here uo uhe toeioisgee york which uhfy yin fought here iate thus ear ro mocky advancee it is rather for ur uo ae here dedicatee uo uhe great task renagogoi adfore ur that from there honoree deae ye uale increasee devotion uo that aause for which uhfy here iate uhe last eull measure oe devotion that ye here highky resolve that there deae shall oou iate fide io vain that this nation under ioe shall iate azoey birth oe freedom and that ioternmenu oe uhe people ay uhe people for uhe people shall oou perish from uhe earthXÓ#ì#' €±€€‚‚ÿIf you were a student of American history, maybe you could recognize it, but you certainly couldn't read it. Somehow we had to distinguish between the different dictionary words that got hashed to the same code. We could factor in the relative popularity of each word, which would help, but this would still make too many mistakes.At this point I started working with Harald Rau on the project, who proved a great collaborator for two reasons. First, he was a bright and peristent graduate student. Second, as a native German speaker he would believe every lie I told him about English grammar.°bm#œ #N j€É€€†"€‡€€‚ç[Ü{€†"€‰€‚‚ÿ  Figure: The phases of the telephone code reconstruction process  Harald built up a phone code reconstruction program on the lines of Figure . It worked on the input one sentence at a time, identifying dictionary words that matched each code string. The key problem was how to incorporate the grammatical constraints.``We can get good word-use frequencies and grammatical information using this big text database called the Brown Corpus. It contains thousands of typical English sentences, each of which is parsed according to parts of speech. But how do we factor it all in?'' Harald asked.øœì#”#; D€{€€‚€€‚€€€€‚ÿ``Let's try to think about it as a graph problem,'' I suggested.``Graph problem? What graph problem? Where is there even a graph?''``Think of a sentence as a list of phone tokens, each representing a word in the sentence. For each phone token, we have a list of words from the dictionary that match it. How can we choose which one is right? Each possible sentence interpretation can be thought of as a path in a graph. The vertices of this graph will be the complete set of possible word choices. There will be an edge from a possible choice for the ith word to each possible choice for the (i+1)st word. The cheapest path across this graph is the right interpretation of the sentence.''–]œ #6A#9 @€œ€€†"€ˆ€€‚‚‚ÿ  Figure: The minimum-cost path through the graph defines the best interpretation for a sentence  ``But all the paths look the same. They have the same number of edges. Wait. Now I see! To make the paths different, we have to weight the edges.''``Exactly! The cost of an edge will reflect how lik”#6A#ӊ"ely it is that we will want to travel through the given pair of words. Maybe we can count how often that pair of words occurred together in previous texts. Or we can weight by what part of speech each word is. Maybe nouns don't like to be next to nouns as much as they like being next to verbs.'''ü”#]C#+ $€ù€€‚‚‚‚ÿ``It will be hard to keep track of word-pair statistics, since there are so many of them. But we certainly know the frequency of each word. How can we factor that into things?''``We can pay a cost for walking through a particular vertex that depends upon the frequency of the word. Our best sentence will be given by the shortest path across the graph.''  ``But how do we figure out the relative weights of these factors?''``Well, try what seems natural to you and then we can experiment with it.''%ô6A#‚D#1 0€é€€çœ—K€‰€‚ÿHarald incorported this shortest-path algorithm. With proper grammatical and statistical constraints, the system performed great, as reported in [RS96]. Look at the Gettysburg Address now, with all the reconstruction errors highlighted:â€]C#dJ#> J€I €Ȁ€€€€€€€€‚ÿFOURSCORE AND SEVEN YEARS AGO OUR FATHERS BROUGHT FORTH ON THIS CONTINENT A NEW NATION CONCEIVED IN LIBERTY AND DEDICATED TO THE PROPOSITION THAT ALL MEN ARE CREATED EQUAL. NOW WE ARE ENGAGED IN A GREAT CIVIL WAR TESTING WHETHER THAT NATION OR ANY NATION SO CONCEIVED AND SO DEDICATED CAN LONG ENDURE. WE ARE MET ON A GREAT BATTLEFIELD OF THAT WAS. WE HAVE COME TO DEDICATE A PORTION OF THAT FIELD AS A FINAL SERVING PLACE FOR THOSE WHO HERE HAVE THEIR LIVES THAT THE NATION MIGHT LIVE. IT IS ALTOGETHER FITTING AND PROPER THAT WE SHOULD DO THIS. BUT IN A LARGER SENSE WE CAN NOT DEDICATE WE CAN NOT CONSECRATE WE CAN NOT HALLOW THIS GROUND. THE BRAVE MEN LIVING AND DEAD WHO STRUGGLED HERE HAVE CONSECRATED IT FAR ABOVE OUR POOR POWER TO ADD OR DETRACT. THE WORLD WILL LITTLE NOTE NOR LONG REMEMBER WHAT WE SAY HERE BUT IT CAN NEVER FORGET WHAT THEY DID HERE. IT IS FOR US THE LIVING RATHER TO BE DEDICATED HERE TO THE UNFINISHED WORK WHICH THEY WHO FOUGHT HERE HAVE THUS FAR SO NOBLY ADVANCED. IT IS RATHER FOR US TO BE HERE DEDICATED TO THE GREAT TASK REMAINING BEFORE US THAT FROM THESE HONORED DEAD WE TAKE INCREASED DEVOTION TO THAT CAUSE FOR WHICH THEY HERE HAVE THE LAST FULL MEASURE OF DEVOTION THAT WE HERE HIGHLY RESOLVE THAT THESE DEAD SHALL NOT HAVE DIED IN VAIN THAT THIS NATION UNDER GOD SHALL HAVE A NEW BIRTH OF FREEDOM AND THAT GOVERNMENT OF THE PEOPLE BY THE PEOPLE FOR THE PEOPLE SHALL NOT PERISH FROM THE EARTH.žl‚D#M#L f€Ý€€†"€‰€€‚çIΕ€†"€‰€‚ÿ  Figure:   Our telephone-code reconstruction system applied to various text samplesWhile we still made a few mistakes, the results are clearly good enough for a variety of applications. Periphonics certainly thought so, for they later licensed our program to incorporate into their products. Figure shows that we were able to reconstruct over 99% of the characters correctly on a megabyte of President Clinton's speeches, so if Bill had phoned them in, we would certainly still be able to understand it. The reconstruction time is fast enough, indeed faster than you can type it in on the phone keypad.!ùdJ#=O#( €ó€€€‚ÿThe constraints associated with many different pattern recognition problems can be formulated as shortest path problems in graphs. In fact, there is a particularly convenient dynamic programming solution for these problems known as the Viterbi algorithm, which is used in speech and handwriting recognition systems. Despite the fancy name, all the Viterbi algorithm is doing is solving a shortest path problem. Hunting for a graph formulation for any given problem is always a good way to proceed.   )M#fO#& €€€‚ÿŽ³=O#&# Ёw€€‚çW9ۀ†"€ ‰€çö7ۀ†"€ ‰€ç*9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ¿Û/€†"€fO#&#ӊ"‰€æ ¿Û/€†"€‰€æ5¿Û/€†"€‰€€çW9ۀ‰€€çö7ۀ‰€€ç*9ۀ‰€€‚‚ÿ Next: Exercises Up: Graph Algorithms Previous: War Story: Nothing but AlgorithmsMon Jun 2 23:33:50 EDT 1997)fO#O#& €€€‚ÿµf&#‚#O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1O#5‚#1Uÿÿÿÿÿÿÿÿ­ÿÿÿÿ5‚#Y‚#$‚#Y‚#" €€€ ÿ: 5‚#“‚#1™*ÿÿÿÿÿÿÿÿ®“‚#Â#ñ $Exercises0 Y‚#Â## €€€‚ÿExercises ®©“‚#q„# ؁c€€çdn_ɀ†"€ ‰€çö7ۀ†"€ ‰€ç+9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ,¿Û/€†"€‰€æ-¿Û/€†"€‰€æ.¿Û/€†"€‰€€çdn_ɀ‰€€çö7ۀ‰€€ç+9ۀ‰€‚€‚€‚ÿ Next: Implementation Challenges Up: Graph Algorithms Previous: War Story: Dialing forExercises ØÂ#€…#7 <€±€Ȁ€€€€€€‚ÿ1. Present correct and efficient algorithms to convert between the following graph data structures, for an undirected graph G with n vertices and m edges. You must give the time complexity of each algorithm.°Rq„#0‡#^ Š€¥€‘€€‚€€€€€€€€€€€€€€€€€€‚‚ÿ1. Convert from an adjacency matrix to adjacency lists.2. Convert from an adjacency list to an incidence matrix. An incidence matrix M has a row for each vertex and a column for each edge, such that M[i,j]=1 if vertex i is part of edge j, otherwise M[i,j] = 0.3. Convert from an incidence matrix to adjacency lists.Å€…#@Š#K d€‹€Ȁ‚‚€€€€€€€€€€€€‚ÿ2. Is the path between a pair of vertices in a minimum spanning tree necessarily a shortest path between the two vertices in the full graph? Give a proof or a counterexample.3. Assume that all edges in the graph have distinct edge weights (i.e. no pair of edges have the same weight). Is the path between a pair of vertices in a minimum spanning tree necessarily a shortest path between the two vertices in the full graph? Give a proof or a counterexample.4. Suppose G is a connected undirected graph. An edge e whose removal disconnects the graph is called a bridge. Must every bridge e be an edge in a depth-first search tree of G, or can e be a back edge? Give a proof or a counterexample.P0‡#‹#7 <€3€Ȁ€€€€€€‚ÿ5. (*) In breadth-first and depth-first search, an undiscovered node is marked discovered when it is first encountered, and marked completely-explored when it has been completely searched. At any given moment, several nodes might be simultaneously in the discovered state. À@Š#Œ#M h€ƒ€Ȁ€€€€€€€€†"€π€‚ÿ(a) Describe a graph on n vertices and a particular starting vertex v such that during a breadth-first search starting from v, nodes are simultaneously in the discovered state. Ÿ‹#š#M h€€Ȁ€€€€€€€€†"€π€‚ÿ(b) Describe a graph on n vertices and a particular starting vertex v such that during a depth-first search starting from v, nodes are simultaneously in the discovered state.o Œ##À#c ”€€Ȁ€€€€€€€€†"€π€†"€π€€€‚‚ÿ(c) Describe a graph on n vertices and a particular starting vertex v such that at some point during a depth-first search starting from v, nodes remain undiscovered, while nodes have been completely-explored. (Note, there may also be discovered nodes.)6. Given the pre-order and in-order traversals of a binary tree, is it possible to reconstruct the tree? If so, sketch an algorithm to do it. If not, give a counterexample. Repeat the problem if you are given the pre-order and pš##À#Y‚#ost-order traversals.<š#_Á#: B€€ȀçUœX€†"€‰€‚ÿ7. Suppose an arithmetic expression is given as a tree. Each leaf is an integer and each internal node is one of the standard arithmetical operations (+,-,*,/). For example, the expression 2+3*4+(3*4)/5 could be represented by the tree in Figure (a).wB#À#ÖÁ#5 :€†€Ȁ†"€Š€€‚ÿ  Figure: Expression 2+3*4+(3*4)/5 as a tree and a DAG.  Ïc_Á#¥Ä#l Š€É€Ȁ€€€€€€‚çUœX€†"€‰€€€€€€€€€€€‚ÿGive an O(n) algorithm for evaluating such an expression, where there are n nodes in the tree.8. (*) Suppose an arithmetic expression is given as a DAG (directed acyclic graph) with common subexpressions removed. Each leaf is an integer and each internal node is one of the standard arithmetical operations (+,-,*,/). For example, the expression 2+3*4+(3*4)/5 could be represented by the DAG in Figure (b). Give an O(n+m) algorithm for evaluating such a DAG, where there are n nodes and m edges in the DAG. Hint: modify an algorithm for the tree case to achieve the desired efficiency.d²ÖÁ# Ç#² 2g€Ȁ€€€€€€€€€€€€€€€€€€€€†"€‹€€€€€€€€€€€€€€€€€€€€€€€€‚ÿ9. (*) Given an undirected graph G with n vertices and m edges, and an integer k, give an O(m+n) algorithm that finds the maximum induced subgraph H of G such that each vertex in H has degree , or prove that no such graph exists. An induced subgraph F=(U,R) of a graph G=(V,E) is a subset of U of the vertices V of G, and all edges R of G such that both vertices of each edge are in U.fæ¥Ä#oÉ#€ ΀Í€Ȁ€€€€€€€€€€€€€€€€€€€€€€‚€€€€€€€€‚ÿ10. (*) An articulation vertex of a graph G is a vertex whose deletion disconnects G. Let G be a graph with n vertices and m edges. Give a simple O(n+m) algorithm for finding a vertex of G that is not an articulation vertex, i.e. whose deletion does not disconnect G.11. (*) Following up on the previous problem, give an O(n+m) algorithm that finds a deletion order for the n vertices such that no deletion disconnects the graph. (Hint: think DFS/BFS.)™õ Ç#Í#€ ï€Ȁ€€€€€€€€†"€Q€€†"€Œ‚€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿ12. (*) Let G be a weighted directed graph with n vertices and m edges, where all edges have positive weight. A directed cycle is a directed path that starts and ends at the same vertex and contains at least one edge. Give an algorithm to find a directed cycle in G of minimum total weight. Partial credit will be given for an algorithm.13. (*) Suppose we are given the minimum spanning tree T of a given graph G (with n vertices and m edges) and a new edge e=(u,v) of weight w that we will add to G. Give an efficient algorithm to find the minimum spanning tree of the graph G + e. Your algorithm should run in O(n) time to receive full credit, although slower but correct algorithms will receive partial credit.LýoÉ#TÎ#O l€û€Ȁ€€€€€€€€€€€€€€‚ÿ14. (*) (a) Let T be a minimum spanning tree of a weighted graph G. Construct a new graph G' by adding a weight of k to every edge of G. Do the edges of T form a minimum spanning tree of G'? Prove the statement or give a counterexample.'‘Í#‡$– ú€)€Ȁ†"€€€€€€€€€€€€€€€€€‚†"€Ž€€†"€€€€€€€€€€€‚ÿ(b) Let describe a shortest weighted path between vertices and t of a weighted graph G. Construct a new graph G' by adding a weight of k to every edge of G. Does P describe a shortest path from to t in G'? Prove the statement or give a counterexample.15. (*TÎ#‡$Y‚#) In certain graph problems, vertices have can have weights instead of or in addition to the weights of edges. Let be the cost of vertex v, and the cost of the edge (x,y). This problem is concerned with finding the cheapest path between vertices a and b in a graph G. The cost of a path is the sum of the costs of the edges and vertices encountered on the path.×dTÎ#^$s Ž€Ï€‘€€€€†"€†"€‘†"€’€€€€€€‚€€€€€€€€‚ÿu Suppose that each edge in the graph has a weight of zero (while non-edges have a cost of ). Assume that for all vertices (i.e. all vertices have the same cost). Give an efficient algorithm to find the cheapest path from a to b and its time complexity. For partial credit, give a less efficient but correct algorithm.u Now suppose that the vertex costs are not constant (but are all positive) and the edge costs remain as above. Give an efficient algorithm to find the cheapest path from a to b and its time complexity. For partial credit, give a less efficient but correct algorithm.;‡$™$; D€€‘€€€€€€€€€‚ÿu Now suppose that both the edge and vertex costs are not constant (but are all positive). Give an efficient algorithm to find the cheapest path from a to b and its time complexity. For partial credit, give a less efficient but correct algorithm.Dÿ^$Ý$E X€ÿ€Ȁ€€€€‚€€€€€€‚ÿ16. (*) Devise and analyze an algorithm that takes a weighted graph G and finds the smallest change in the cost of a non-MST edge that causes a change in the minimum spanning tree of G. Your algorithm must be correct and run in polynomial time.17. An arborescence of a directed graph G is a rooted tree such that there is a directed path from the root to every other vertex in the graph. Give an efficient and correct algorithm to test whether G contains an arborescence, and its time complexity.ã™$ú$: B€É€Ȁç-€†"€‰€‚ÿ18. (**) The war story of Section describes an algorithm for constructing the dual graph of the triangulation efficiently, although it does not guarantee linear time. Give a worst-case linear algorithm for the problem.,Ý$& $' € €€€‚ÿ)ú$O $& €€€‚ÿÄÃ& $ $ Ё—€€‚çdn_ɀ†"€ ‰€çö7ۀ†"€ ‰€ç+9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ/¿Û/€†"€‰€æ0¿Û/€†"€‰€æ1¿Û/€†"€‰€€çdn_ɀ‰€€çö7ۀ‰€€ç+9ۀ‰€€‚‚ÿ Next: Implementation Challenges Up: Graph Algorithms Previous: War Story: Dialing for AlgorithmsMon Jun 2 23:33:50 EDT 1997)O $< $& €€€‚ÿµf $ñ $O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1< $" $1Uÿÿÿÿÿÿÿÿ¯ÿÿÿÿ" $F $$ñ $F $" €€€ ÿ= " $ƒ $1‡ÿÿÿÿÿÿÿÿ°ƒ $¶ $·ƒ$Backtracking3F $¶ $# € €€‚ÿBacktracking kdƒ $-@$ ܁Ù€€çO9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ2¿Û/€†"€‰€æ3¿Û/€†"€‰€æ4¿Û/€†"€‰€€çO9ۀ‰€€ç÷7ۀ‰€€ç÷7ۀ‰€‚€‚€‚‚ÿ Next: Constructing All Subsets Up: Combinatorial Search and Heuristic Previous: Combinatorial Search and HeuristicBacktracking Backtracking is a systematic way to go through all the possible configurations of a space. These configurations may be all possible arrangements of objects (permutations) or all possible ways of building a collection of them (subsets). Other applications may demand enumerating all spanning trees of a graph, all paths between two vertices, or all possible ways to partition the¶ $-@$F $ vertices into color classes.  g!¶ $”B$F Z€I€€†"€“†"€”†"€•€€‚ÿWhat these problems have in common is that we must generate each one of the possible configurations exactly once. Avoiding both repetitions and missing configurations means that we must define a systematic generation order among the possible configurations. In combinatorial search, we represent our configurations by a vector , where each element is selected from an ordered set of possible candidates for position i. As shown below, this representation is general enough to encode most any type of combinatorial object naturally.  Mø-@$áD$U x€ù€€€€†"€–†"€—†"€˜€€†"€˜‚ÿThe search procedure works by growing solutions one element at a time. At each step in the search, we will have constructed a partial solution with elements fixed for the first k elements of the vector, where . From this partial solution , we will construct the set of possible candidates for the (k+1)st position. We will then try to extend the partial solution by adding the next element from . So long as the extension yields a longer partial solution, we continue to try to extend it.n(”B$OF$F Z€W€€†"€˜€€†"€™†"€š‚ÿHowever, at some point, might be empty, meaning that there is no legal way to extend the current partial solution. If so, we must backtrack, and replace , the last item in the solution value, with the next candidate in . It is this backtracking step that gives the procedure its name:RáD$_H$Ÿ J³€€‚€€‚†"€Ȁ€‚€€‚€€‚†"€›‚†"€™†"€š‚†"€œ‚†"€‚€€€€‚†"€š€€€€‚ÿ Backtrack(A) Compute , the set of candidate first elements of solution A. k = 1 while k > 0 do while do (*advance*) = the next element from if is a solution, report it. k = k + 1 compute , the set of candidate kth elements of solution A.fOF$ÅJ$P n€-€€€€€€‚€€€€€€€€€‚ÿ k = k - 1 (*backtrack*) Backtracking constructs a tree of partial solutions, where each vertex is a partial solution. There is an edge from x to y if node y was created by advancing from x. This tree of partial solutions provides an alternative way to think about backtracking, for the process of constructing the solutions corresponds exactly to doing a depth-first traversal of the backtrack tree. Viewing backtracking as depth-first search yields a natural recursive implementation of the basic algorithm:  È_H$N$³ 49€€‚€€‚†"€‚‚€€€€‚†"€š‚†"€›‚†"€™†"€š‚†"€š†"€ž‚€€‚€€€€€‚ÿ Backtrack-DFS(A,k) if is a solution, report it. else k = k +1 compute while do = an element in = Backtrack(a,k) Although a breadth-first search could also be used to enumerate all solutions, depth-first search is greatly preferred because of the amount of storage required. In depth-first search, the current state of the search is completely represented by the path from the root to the current search node, which requires space proportional to the height of the tree. In breadth-first search, the queue stores all the nodes at the current level, which is proportional to the width of the search tree. For most interesting problems, the width of the tree will grow exponentially in its height.ØÅJ$O$( €±€€€‚ÿTo really understand how backtracking works, you must see how such objects as permutations and subsets can be constructed by defining the right state spaces. Examples of several state spaces are described below.)N$¶O$& €€€‚ÿ&O$ÜO$# €€€‚ÿÁq¶O$©€$P p€â€ȀçÜO$©€$F $âg`€‰€‚€çcpn¿€‰€‚€çÄþt€‰€‚ÿu Constructing All Subsets u Constructing All Permutations u Constructing All Paths in a Graph&ÜO$π$# €€€‚ÿ)©€$ø€$& €€€‚ÿáàπ$ق$ ЁÑ€€‚çO9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ`¿Û/€†"€‰€æW¿Û/€†"€‰€æX¿Û/€†"€‰€€çO9ۀ‰€€ç÷7ۀ‰€€ç÷7ۀ‰€€‚‚ÿ Next: Constructing All Subsets Up: Combinatorial Search and Heuristic Previous: Combinatorial Search and Heuristic AlgorithmsMon Jun 2 23:33:50 EDT 1997)ø€$ƒ$& €€€‚ÿµfق$·ƒ$O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ƒ$èƒ$1Uÿÿÿÿÿÿÿÿ±ÿÿÿÿèƒ$ „$$·ƒ$ „$" €€€ ÿIèƒ$U„$1N ÿÿÿÿÿÿÿÿ²U„$”„$±$Constructing All Subsets? „$”„$# €8€€‚ÿConstructing All Subsets ©bU„$=ˆ$G \‚Ý€€çP9ۀ†"€ ‰€çN9ۀ†"€ ‰€çN9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æY¿Û/€†"€‰€æZ¿Û/€†"€‰€æ[¿Û/€†"€‰€€çP9ۀ‰€€çN9ۀ‰€€çN9ۀ‰€‚€ ‚€€€†"€w€€†"€Ÿ†"€ €€€€†"€U€€‚ÿ Next: Constructing All Permutations Up: Backtracking Previous: BacktrackingConstructing All SubsetsTo design a suitable state space for representing a collection of combinatorial objects, it is important to know how many objects you will need to represent. How many subsets are there of an n-element set, say the integers ? There are exactly two subsets for n=1 namely and , four subsets for n=2, and eight subsets for n=3. Since each new element doubles the number of possibilities, there are subsets of n elements.  «”„$Z‹$r ²€a€€†"€U€€†"€”€€†"€¡€€†"€¢‚†"€p€€€€‚ÿEach subset is described by stating which elements are in it. Therefore, to construct all subsets, we can set up an array/vector of n cells, where the value of is either true or false, signifying whether the ith item is or is not in the given subset. To use the notation of the general backtrack algorithm, , and A is a solution whenever .Using this state space representation, the backtracking algorithm constructs the following sequence of partial solutions in finding the subsets of . Final solutions, i.e. complete subsets, are marked with a *. False choices correspond to dashes in the partial solution, while true in position i is denoted by i itself:ýµ=ˆ$WŒ$H ^€o€€†"€£‚ç `4"€†"€‰€€‚ÿTrace through this example carefully to make sure you understand the backtracking procedure. The problem of generating subsets is more thoroughly discussed in Section .)Z‹$€Œ$& €€€‚ÿS+WŒ$ӌ$( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)€Œ$üŒ$& €€€‚ÿµfӌ$±$O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1üŒ$â$1Uÿÿÿÿÿÿÿÿ³ÿÿÿÿâ$Ž$$±$Ž$" €€€ ÿNâ$TŽ$1> ÿÿÿÿÿÿÿÿŽTŽ$˜Ž$ŒÆ$Constructing All PermutationsD!Ž$˜Ž$# €B€€‚ÿConstructing All Permutations XTŽ$1Â$5 8‚É€€çQ9ۀ†"€ ‰€çN9ۀ†"€ ‰€çO9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ\¿Û/€†"€‰€æ]¿Û/€†"€‰€æ^¿Û/€†"€‰€€çQ9ۀ‰€€çN9ۀ‰€€çO9ۀ‰€‚€ ‚€€€†"€w†"€€€€†"€€†"€¥‚ÿ ˜Ž$1Â$Ž$Next: Constructing All Paths in Up: Backtracking Previous: Constructing All SubsetsConstructing All PermutationsTo design a suitable state space for representing permutations, we start by counting them. There are n distinct choices for the value of the first element of a permutation of . Once we have fixed this value of , there are n-1 candidates remaining for the second position, since we can have any value except (repetitions are forbidden). Repeating this argument yields a total of distinct permutations.   ˜Ž$ŸÄ$m š€E€€€€€€€€€€€€€€†"€Š€€€€€€†"€p‚ÿThis counting argument suggests a suitable representation. To construct all n! permutations, set up an array/vector A of n cells. The set of candidates for the ith position will be the set of elements that have not appeared in the (i-1) elements of the partial solution, corresponding to the first i-1 elements of the permutation. To use the notation of the general backtrack algorithm, . The vector A contains a full solution whenever k = n+1. This representation generates the permutations of in the following order:€]1Â$bÅ$G ^€Ÿ€€†"€§‚ç©Ñ-À†"€‰€€‚ÿThe problem of generating permutations is more thoroughly discussed in Section .)ŸÄ$‹Å$& €€€‚ÿS+bÅ$ÞÅ$( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)‹Å$Æ$& €€€‚ÿµfÞÅ$ŒÆ$O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Æ$íÆ$1UÿÿÿÿÿÿÿÿµÿÿÿÿíÆ$Ç$$ŒÆ$Ç$" €€€ ÿR!íÆ$cÇ$1Bÿÿÿÿÿÿÿÿ¶cÇ$«Ç$ŒÏ$Constructing All Paths in a GraphH%Ç$«Ç$# €J€€‚ÿConstructing All Paths in a Graph )cÇ$ÔÊ$  äM€€çR9ۀ†"€ ‰€çN9ۀ†"€ ‰€çP9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ_¿Û/€†"€‰€拿Û/€†"€‰€悿Û/€†"€‰€€çR9ۀ‰€€çN9ۀ‰€€çP9ۀ‰€‚€ ‚€€€‚ÿ Next: Search Pruning Up: Backtracking Previous: Constructing All PermutationsConstructing All Paths in a GraphEnumerating all the simple paths from to t through a given graph is a somewhat more complicated problem than listing permutations or subsets. Unlike the earlier problems, there is no explicit formula that counts the number of solutions as a function of the number of edges or vertices, because the number of paths depends upon the structure of the graph.  Žè«Ç$bÎ$Š ß€€†"€š€€€€‚€€†"€Ȇ"€©€€€€†"€˜†"€™€€†"€ª€€€€çœþt€†"€‰€€‚ÿ  Figure: The search tree enumerating all simple paths from vertex 1 in the graph  Since the starting point of any path from to t is always , must be . The set of possible candidates for the second position are the vertices v such that (,v) is an edge of the graph, for the path wanders from vertex to vertex using edges to define the legal steps. In general, consists of the set of vertices adjacent to that have not been used in the partial solution A. We can report a successful path whenever . The solution vector A must have room for all n vertices, although most paths are likely to be shorter than this. Figure shows the search tree giving all paths from a particular vertex in an example graph.)ÔÊ$‹Î$& €€€‚ÿS+bÎ$ÞÎ$( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)‹Î$Ï$& €€€‚ÿµfÞÎ$ŒÏ$O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Ï$ %1Uÿÿÿÿÿÿÿÿ·ÿÿÿÿ %0%ŒÏ$ %ŒÏ$$ŒÏ$0%" €€€ ÿ? %o%1ªÿÿÿÿÿÿÿÿžo%€%«A%Search Pruning50%€%# €$€€‚ÿSearch Pruning uXo%% ‚Á€€çS9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çQ9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€惿Û/€†"€‰€愿Û/€†"€‰€慿Û/€†"€‰€€çS9ۀ‰€€ç÷7ۀ‰€€çQ9ۀ‰€‚€‚€€€€€€€€€‚ÿ Next: Bandwidth Minimization Up: Combinatorial Search and Heuristic Previous: Constructing All Paths inSearch PruningBacktracking ensures correctness by enumerating all possibilities. For example, a correct algorithm to find the optimal traveling salesman tour could enumerate all n! permutations of n vertices of the graph and selecting the best one. For each permutation, we could check whether each of the n edges implied in the tour really exists in the graph G, and if so, sum the weights of these edges together.2€%©%^ Š€o€€†"€m†"€l€€€€†"€«†"€«†"€¬‚ÿFor most graphs, however, it would be pointless to construct all the permutations first and then analyze them later. Suppose we started our search from vertex , and it happened that edge was not in G. Enumerating all the (n-2)! permutations beginning with would be a complete waste of effort. Much better would be to prune the search after and continue next with . By carefully restricting the set of next elements to reflect only the moves that are legal from the current partial configuration, we reduce the search complexity significantly.  žW%a %a €·€€€€†"€­†"€®€€€€†"€¯†"€®€€‚ÿPruning is the technique of cutting off search the instant we have established that this partial solution cannot be extended into the solution that we want. For example, in our traveling salesman search program, we seek the cheapest tour that visits all vertices before returning to its starting position. Suppose that in the course of our search we find a tour t whose cost is . As the search continues, perhaps we will find a partial solution , where k < n and the sum of the edges on this partial tour is . Can there be any reason to continue exploring this node any further? No, assuming all edges have positive cost, because any tour with the prefix will have cost greater than tour t, and hence is doomed to be non-optimal. Cutting away such failed partial tours as soon as possible can have an enormous impact on running time.  aý©%Â%d –€€€†"€m†"€n†"€n€€€€€€†"€m€€€‚ÿExploiting symmetry is a third avenue for reducing combinatorial search. It is clearly wasteful to evaluate the same candidate solution more than once, because we will get the exact same answer each time we consider it. Pruning away partial solutions identical to those previously considered requires recognizing underlying symmetries in the search space. For example, consider the state of our search for an optimal TSP tour after we have tried all partial positions beginning with . Can it pay to continue the search with partial solutions beginning with ? No. Any tour starting and ending at can be viewed as starting and ending at or any other vertex, for these tours are cycles. There are thus only (n-1)! distinct tours on n vertices, not n!. By restricting the first element of the tour to always be , we save a factor of n in time without missing any interesting solutions. Detecting such symmetries can be subtle, but once identified they can usually be easily exploited by a search program.)a %ë%& €€€‚ÿÖÕÂ%Í@% Ё»€€‚çS9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çQ9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€憿Û/€†"€‰€懿Û/€†"€‰€房Û/€†"€‰€€çS9ۀ‰€€ç÷7ۀ‰€€çQ9ۀ‰€€‚‚ÿ ë%Í@%0% Next: Bandwidth Minimization Up: Combinatorial Search and Heuristic Previous: Constructing All Paths in AlgorithmsMon Jun 2 23:33:50 EDT 1997)ë%ö@%& €€€‚ÿµfÍ@%«A%O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ö@%ÜA%1Uÿÿÿÿÿÿÿÿ¹ÿÿÿÿÜA%B%$«A%B%" €€€ ÿGÜA%GB%1 ÿÿÿÿÿÿÿÿºGB%„B%‹%Bandwidth Minimization=B%„B%# €4€€‚ÿBandwidth Minimization 7"GB%»D% øW€€çT9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çR9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€承Û/€†"€‰€抿Û/€†"€‰€涿Û/€†"€‰€€çT9ۀ‰€€ç÷7ۀ‰€€çR9ۀ‰€‚€‚€†"€°€€‚ÿ Next: War Story: Covering Chessboards Up: Combinatorial Search and Heuristic Previous: Search PruningBandwidth Minimization    Figure: A pretty bandwidth-4 layout of a binary tree atop an ugly bandwidth-3 layoutž/„B%YH%o ¬€c€€€€çYS!€†"€‰€‚€€€€€€€€ç~‚Ü€†"€‰€‚ÿTo better demonstrate the power of pruning and symmetry detection, let's apply these ideas to producing a search program that solves the bandwidth minimization problem, discussed in detail in catalog Section . I annually run competitions for the fastest bandwidth-minimization program for students in my algorithms courses; the timings below are drawn from these experiences.  The bandwidth problem takes as input a graph G, with n vertices and m edges. The goal is to find a permutation of the vertices on the line that minimizes the maximum length of any edge. Figure gives two distinct layouts of a complete binary tree on 15 vertices. The clean, neat layout on the top has a longest edge of length 4, but the seemingly cramped layout on the bottom realizes the optimal bandwidth of 3."ì»D%{K%6 :€Û€€‚€€†"€±‚ÿThe bandwidth problem has a variety of applications, including circuit layout, linear algebra, and optimizing memory usage in hypertext documents. The problem is NP-complete, which implies that no polynomial time worst-case algorithm is known for the problem. It remains NP-complete even for very restricted classes of trees.Since the bandwidth problem seeks a particular permutation, a backtracking program that iterates through all the n! possible permutations and computes the length of the longest edge for each gives a straightforward algorithm. Depending upon how well it is programmed, and how fast a machine it is running on, such an algorithm can expect to solve instances of approximately 8 to 12 vertices within one CPU minute.²~YH%-N%4 6€ÿ€€€€†"€²‚ÿTo speed up this search, we can try to exploit symmetry. For any permutation p, its reverse permutation will realize the exact same bandwidth, since the length of each edge is the same. Hence we can immediately eliminate half of our search space. The reverse copies are easily removed by placing the leftmost and rightmost elements of the permutation as the first two elements of the vector and then pruning if . Because we are dealing with an exponential search, removing a single factor of two can only be of limited usefulness. Such symmetry elimination might add one to the size of the problem we can do within one CPU minute.w{K%°€%[ „€A€€€€†"€³†"€Ž†"€®€€€€†"€³‚ÿFor more serious speedups, we need to prune partial solutions. Say we have found a permutation p that yields a longest edge of . By definition, . Suppose that among the elements in a partial layout , where k < n, there is an edge that is at least in length. Can this partial solution expand to provide a better bandwidth solution? Of course not! By pruning the-N%°€%B% search the instant we have created a long edge, typical instances of 15 to 20 vertices can be solved in one CPU minute, thus providing a substantial improvement.Ä™-N%tƒ%+ $€3€€€€‚ÿEfforts to further improve the search algorithm must strive for even greater pruning. By using a heuristic method to find the best solution we can before starting to search, we save time by realizing early length cutoffs. In fact, most of the effort in a combinatorial search is typically spent after the optimal solution is found, in the course of proving that no better answer exists. By observing that the optimal bandwidth solution must always be at least half the degree of any vertex (think about the incident edges), we have a lower bound on the size of the optimal solution. We can terminate search soon as we find a solution matching the lower bound.ć°€%8†%= H€€€€€€€€€€€‚ÿOne limitation of this pruning strategy is that only partial solutions of length >b can be pruned, where b is the bandwidth of the best solution to date, since we must place b+1 vertices before we can generate any edges of length at least b. To achieve earlier cutoffs, we can alternately fill in the leftmost and rightmost slots of the configuration, instead of always proceeding from the left. This way, whenever there is an edge between a vertex on the left side and a vertex on the right side, this edge is likely long enough to achieve a cutoff. Pruning can easily occur while positioning the second vertex in the solution vector.õÍtƒ%-ˆ%( €›€€€‚ÿUsing these enhancements, top-notch programs are capable of solving typical problems on up to 30 vertices consistently within one CPU minute, operating literally millions of times faster than unpruned, untuned efforts. The speed difference between the final and initial versions of the program dwarf the difference between a supercomputer and a microcomputer. Clever search algorithms can easily have a bigger impact on performance than expensive hardware.)8†%Vˆ%& €€€‚ÿÔÓ-ˆ%*Š% Ё·€€‚çT9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çR9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ­¿Û/€†"€‰€殿Û/€†"€‰€毿Û/€†"€‰€€çT9ۀ‰€€ç÷7ۀ‰€€çR9ۀ‰€€‚‚ÿ Next: War Story: Covering Chessboards Up: Combinatorial Search and Heuristic Previous: Search Pruning AlgorithmsMon Jun 2 23:33:50 EDT 1997)Vˆ%SŠ%& €€€‚ÿµf*Š%‹%O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1SŠ%9‹%1Uÿÿÿÿÿÿÿÿ»ÿÿÿÿ9‹%]‹%$‹%]‹%" €€€ ÿP9‹%­‹%1#'ÿÿÿÿÿÿÿÿŒ­‹%ó‹%œ &War Story: Covering ChessboardsF#]‹%ó‹%# €F€€‚ÿWar Story: Covering Chessboards yt­‹%l% ؁ù€€çU9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çS9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ°¿Û/€†"€‰€汿Û/€†"€‰€沿Û/€†"€‰€€çU9ۀ‰€€ç÷7ۀ‰€€çS9ۀ‰€‚€‚€‚ÿ Next: Heuristic Methods Up: Combinatorial Search and Heuristic Previous: Bandwidth MinimizationWar Story: Covering ChessboardsEvery researcher dreams of solving a classical problem, one that has remained open and unsolved for over a hundred years. There is something romantic about communicating across the generations, being part of the evolution of science, helping to climb another rung up the ladder of human progress. There is also a pleasant sense of smugness that comes from figuring out how to do something that nobody else could do before you.  …`ó‹%ýÁ%% €Á€€‚ÿThere are several possible reasons why a problem might stay open for such a long period of time. Perhaps the l%ýÁ%]‹%problem is so difficult and profound that it requires a uniquely powerful intellect to solve. A second reason is technological - the ideas or techniques required to solve the problem may not have existed when the problem was first posed. A final possibility is that no one may have cared enough about the problem in the interim to seriously bother with it. Once, I was involved in solving a problem that had been open for over a hundred years. Decide for yourself which reason best explains why.k1l%hÄ%: B€e€€€€€€†"€µ‚ÿChess is a game that has fascinated mankind for thousands of years. In addition, it has inspired a number of combinatorial problems of independent interest. The combinatorial explosion was first recognized in the legend that the inventor of chess demanded as payment one grain of rice for the first square of the board, and twice the amount of the ith square for the (i+1)st square, for a total of 36,893,488,147,419,103,231 grains. In beheading him, the wise king first established pruning as a technique for dealing with the combinatorial explosion.V1ýÁ%ŸÆ%% €c€€‚ÿIn 1849, Kling posed the question of whether all 64 squares on the board can be simultaneously threatened by an arrangement of the eight main pieces on the chess board - the king, queen, two knights, two rooks, and two oppositely colored bishops. Configurations that simultaneously threaten 63 squares have been known for a long time, but whether this was the best possible remained an open problem. This problem seemed ripe for solution by exhaustive combinatorial searching, although whether it was solvable would depend upon the size of the search space.3úhÄ%ñÈ%9 @€ù€€†"€¶†"€·‚‚ÿConsider the 8 main pieces in chess (king, queen, two rooks, two bishops, two knights). How many ways can they be positioned on a chessboard? The trivial bound is positions. Anything much larger than about positions would be unreasonable to search on a modest computer in a modest amount of time.Getting the job done would require significant pruning. The first idea is to remove symmetries. Considering the orthogonal and diagonal symmetries, there are only ten distinct positions for the queen.Œ\ŸÆ%}Í%0 .€»€€†"€ž‚‚ÿOnce the queen is placed, there are 2,080 distinct ways to position a pair of rooks or knights, 64 places to locate the king, and 32 spots for each of the white and black bishops. Thus to perform an exhaustive search, we must test 2,835,349,504,000 distinct positions, still much too large to try.We could use backtracking to construct all of the positions, but we had to find a way to prune the search space significantly if we could hope to finish in our lifetime. Pruning the search meant that we needed a quick way to prove, for a partially filled-in position, that there was no possible way to complete it so as to cover all 64 squares. Suppose we had already placed seven pieces on the board, and together they covered all but 10 squares of the board. Say the remaining piece was the king. Is there any possible position to place the king so that all squares are threatened? The answer must be no, because the king can threaten at most eight squares according to the rules of chess. There can be no reason to bother testing any of the subsequent positions. By pruning these positions, we might win big. ûñÈ%Ï%% €÷€€‚ÿOptimizing this pruning strategy required carefully ordering the evaluation of the pieces. Each piece could threaten a certain maximum number of squares: the queen 27, the king 8, the rook 14, and the bishop 13. To maximize the chances of a cutoff, we would want to insert the pieces in decreasing order of mobility. Whenever the number of unthreatened squares exceeds the sum of the maximum coverage of the unplaced pieces, we can prune. This sum is minimized by using the decreasing order of mobility.%à}Í%Î&E X€Å€€†"€¹†"€º‚€€€€‚ÿWhen we implemented backtracÏ%Î&]‹%k search with this pruning strategy, we found that it eliminated over of the search space. After optimizing our move generation, our program could search over 1,000 positions per second. But this was still too slow, for seconds meant 1,000 days! Although we might further tweak the program to speed it up by an order of magnitude or so, what we really needed was to find a way to prune more nodes.Effective pruning meant eliminating large numbers of positions at a single stroke. Our previous attempt was too weak. What if instead of placing up to eight pieces on the board simultaneously, we placed more than eight pieces. Obviously, the more pieces we placed simultaneously, the less likely it would be that they didn't threaten all 64 squares. But if they didn't cover, all subsets of eight distinct pieces from the set couldn't possibly threaten all squares. The potential existed to eliminate a vast number of positions by pruning a single node.Z#Ï%(&7 <€G€€€€€€€€‚ÿThus the nodes of our search tree corresponded to chessboards that could have any number of pieces, and more than one piece on a square. For a given board, we would distinguish two kinds of attack on a square: strong and weak. The notion of strong attack corresponds to the usual notion of attack in chess. A square is weakly attacked if the square is strongly attacked by some subset of the board, that is, weak attack ignores any possible blocking effects of intervening pieces. All 64 squares can be weakly attacked with eight pieces.0Î&X&+ $€ €€€€‚ÿOur algorithm consists of two passes. The first pass lists all boards such that every square is weakly attacked. The second pass filters the list by considering blocking and reports any boards with n or fewer safe squares. The advantage of separating weak and strong attack computations is that weak attack is faster to compute (no blocking to worry about), and yet the strong attack set is always a subset of the weak attack set. Whenever there was a non-weakly-threatened square, the position could be pruned.T(&è &< F€©€€çw—K€‰€€€‚€‚ÿThis program was efficient enough to complete the search on a machine as slow as a 1988-era IBM PC-RT in under one day. More details of our searching procedure and results appear in our paper [RHS89]. It did not find a single position covering all 64 squares with the bishops on opposite colored squares. However, our program showed that it is possible to cover the board with seven pieces if a queen and a knight can occupy the same square.The take-home lesson of this war story should be clear. Clever pruning can make short work of surprisingly hard combinatorial search problems.)X& && €€€‚ÿÎÍè &ß & Ё«€€‚çU9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çS9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€泿Û/€†"€‰€掿Û/€†"€‰€浿Û/€†"€‰€€çU9ۀ‰€€ç÷7ۀ‰€€çS9ۀ‰€€‚‚ÿ Next: Heuristic Methods Up: Combinatorial Search and Heuristic Previous: Bandwidth Minimization AlgorithmsMon Jun 2 23:33:50 EDT 1997) & && €€€‚ÿµfß &œ &O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 &î &1Uÿÿÿÿÿÿÿÿœÿÿÿÿî &&$œ &&" €€€ ÿBî &T&1q ÿÿÿÿÿÿÿÿŸT&Œ&qH&Heuristic Methods8&Œ&# €*€€‚ÿHeuristic Methods ñT&£A& ‚õ€€çV9ۀ†"€ ‰€ç÷7ۀ†"€ ‰€çT9ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æá¿Û/€†"€‰€æØ¿Û/€†"€‰€æÙ¿Û/€†"€‰€€çV9ۀ‰€€ç÷7ۀ‰€€çT9ۀ‰€‚€‚€çŸ€†"€‰€‚ÿ Next: Simulated Annealing Up: CŒ&£A&&ombinatorial Search and Heuristic Previous: War Story: Covering ChessboardsHeuristic MethodsThe techniques we have discussed thus far seek to find the optimal answer to a combinatorial problem as quickly as possible. Traditional algorithmic methods fail whenever the problem is provably hard (as discussed in Chapter ), or the problem is not clean enough to lead to a nice formulation.  +Œ&ÎC&+ $€€€€€‚ÿHeuristic methods provide a way to approach difficult combinatorial optimization problems. Combinatorial search gives us a method to construct possible solutions and find the best one, given a function that measures how good each candidate solution is. However, there may be no algorithm to find the best solution short of searching all configurations. Heuristic methods such as simulated annealing, genetic algorithms, and neural networks provide general ways to search for good but not optimal solutions.E£A&E&( €;€€€‚ÿIn this section we discuss such heuristic methods. Each of these three techniques relies on a simple model of a real-world physical process. We devote the bulk of our attention to simulated annealing, which is the easiest method to apply in practice, as well as the most reliable.)ÎC&A'œA'# €6€€‚ÿCircuit Board Placement ©€^A'EE' ؁Y€€çÙg_ɀ†"€ ‰€çV9ۀ†"€ ‰€ç×g_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ ÀÛ/€†"€‰€æ ÀÛ/€†"€‰€æpÇÛ/€†"€‰€€çÙg_ɀ‰€€çV9ۀ‰€€ç×g_ɀ‰€‚€‚€‚ÿ Next: Neural Networks Up: Simulated Annealing Previous: Independent SetCircuit Board PlacementIn designing printed circuit boards, we are faced with the problem of positioning modules (typically integrated circuits) on the board. Desired criteria in a layout include (1) minimizing the area or aspect ratio of the board, so that it properly fits within the allotted space, and (2) minimizing the total or longest wire length in connecting the components. Circuit board placement is an example of the kind of messy, multicriterion optimization problems for which simulated annealing is ideally suited.  ÉœA'YH'K d€›€€†"€Ò†"€Ó†"€Ô†"€Õ‚‚ÿFormally, we are given a collection of a rectangular modules , each with associated dimensions . Further, for each pair of modules , we are given the number of wires that must connect the two modules. We seek a placement of the rectangles that minimizes area and wire-length, subject to the constraint that no two rectangles overlap each other.The state space for this problem must describe the positions of each rectangle. To provide a discrete representation, the rectangles can be restricted to lie on vertices of an integer grid. Reasonable transition mechanisms including moving one rectangle to a different location, or swapping the position of two rectangles. A natural cost function would beQàEE'ªJ'q °€Ë€€†"€Ö‚†"€×†"€Ø†"€Ù†"€Ù‚ç ŠK€‰€çž;C€‰€€‚ÿwhere , , and are constants governing the impact of these components on the cost function. Presumably, should be an inverse function of temperature, so after gross placement it adjusts the rectangle positions so they are distinct.Simulated annealing performs well on such module placement problems. Indeed, a similar application appeared in the original paper on simulated annealing [KGV83]. More details on these and other applications appear in [AK89].)YH'ÓJ'& €€€‚ÿS+ªJ'&K'( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)ÓJ'OK'& €€€‚ÿµf&K'L'O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1OK'5L'1UÿÿÿÿÿÿÿÿÉÿÿÿÿ5L'YL'$L'YL'" €€€ ÿ@5L'™L'1]ÿÿÿÿÿÿÿÿʙL'ÏL'OŠ'Neural Networks6YL'ÏL'# €&€€‚ÿNeural Networks ‚{™L'QO' ܁€€çÚg_ɀ†"€ ‰€çU9ۀ†"€ ‰€çØg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€ægÇÛ/€†"€‰€æhÇÛ/€†"€‰€æiÇÛ/€†"€‰€€çÚg_ɀ‰€€çU9ۀ‰€€çØg_ɀ‰€‚€ ‚€‚‚ÿ Next: Genetic Algorithms Up: Heuristic Methods Previous: Circuit Board PlacementNeural Networks Neural networks are a computational paradigm inspired by the architecture of the human brain. The intuition is that since brains are good at solving problems, machines built in the same way should be, too.  ¢uÏL'ÿ'- (€ë€€€€‚‚ÿThe basic computational component of the brain is a neuron, a simple unit that produces a non-linear, weighted sum of its inputsQO'ÿ'YL', which are connections from other neurons. Neural networks are weighted digraphs with neurons as vertices and weights on edges denoting the connection strength of the pair.Brains are very good at learning and recognizing certain patterns. Learning in brains seems to work by adding connections between different pairs of neurons and changing the strengths of the connections. Modifying connection strength in response to training examples provides a natural way to ``teach'' a neural network.ßžQO'ޅ'' €q€€‚‚ÿAlthough there have been attempts to apply neural networks to solving combinatorial optimization problems, the successes have been rather limited. Simulated annealing is a much more straightforward and efficient approach to optimization.Neural networks have been more successful in classification and forecasting applications, such as optical character recognition, gene prediction, and stock-market time-series prediction. A set of features for the given patterns is selected, and each training example is represented in terms of its features. The network is trained on a series of positive and negative examples, with the strengths of the connections adjusted to recognize these examples. Output cells for each class of item are provided and the strength of these cells on a given input used to determine the classification. Once the network is trained, feature vectors corresponding to unknown items can be entered and a classification made.  «nÿ'‰‡'= H€ß€€çºB¶v€†"€‰€€‚ÿBecause neural networks are black boxes, with the strength of edges adjusted only by the training examples, there is usually no way to figure out exactly why they are making the decisions that they are. A particularly amusing instance where this led to trouble is reported in Section . Still, they can be useful in certain pattern-recognition applications.)ޅ'²‡'& €€€‚ÿ¿Ÿ‰‡'q‰' Ё€€‚çÚg_ɀ†"€ ‰€çU9ۀ†"€ ‰€çØg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æjÇÛ/€†"€‰€ækÇÛ/€†"€‰€ælÇÛ/€†"€‰€€çÚg_ɀ‰€€çU9ۀ‰€€çØg_ɀ‰€€‚‚ÿ Next: Genetic Algorithms Up: Heuristic Methods Previous: Circuit Board Placement AlgorithmsMon Jun 2 23:33:50 EDT 1997)²‡'š‰'& €€€‚ÿµfq‰'OŠ'O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1š‰'€Š'1UÿÿÿÿÿÿÿÿËÿÿÿÿ€Š'€Š'$OŠ'€Š'" €€€ ÿC€Š'çŠ'1È ÿÿÿÿÿÿÿÿÌçŠ' ‹',È'Genetic Algorithms9€Š' ‹'# €,€€‚ÿGenetic Algorithms ­ŠçŠ'͎' ܁]€€çÛg_ɀ†"€ ‰€çU9ۀ†"€ ‰€çÙg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æmÇÛ/€†"€‰€ænÇÛ/€†"€‰€æoÇÛ/€†"€‰€€çÛg_ɀ‰€€çU9ۀ‰€€çÙg_ɀ‰€‚€ ‚€‚‚ÿ Next: War Story: Annealing Arrays Up: Heuristic Methods Previous: Neural NetworksGenetic Algorithms Genetic algorithms draw their inspiration from evolution and natural selection. Through the process of natural selection, organisms adapt to optimize their chances for survival in a given environment. Random mutations occur to the genetic description of an organism, which is then passed on to its children. Should a mutation prove helpful, these children are more likely to survive to reproduce. Should it be harmful, these children are less likely to reproduce, so the bad trait will die with them.  à ‹'ÞÀ'% €Á€€‚ÿGenetic algorithms maintain a ``population'' of solution candidates for the given problem. Elements are drawn at random from this population and allowed to ``reproduce'', by combining some aspects of the two parent solutions. The probability that an element is chosen ͎'ÞÀ'€Š'to reproduce is based on its ``fitness'', essentially a function of the cost of the solution it represents. Eventually, unfit elements die from the population, to be replaced by successful-solution offspring.J%͎'(Ä'% €K€€‚ÿThe idea behind genetic algorithms is extremely appealing. However, they just don't seem to work as well on practical combinatorial optimization problems as simulated annealing does. There are two primary reasons for this. First, it is quite unnatural to model most applications in terms of genetic operators like mutation and crossover on bit strings. The pseudobiology adds another level of complexity between you and your problem. Second, genetic algorithms take a very long time on non-trivial problems. The crossover and mutation operations make no real use of problem-specific structure, so a large fraction of transitions lead to inferior solutions, and convergence is slow. Indeed, the analogy with evolution, where significant improvements require millions of years, can be quite appropriate.=ÞÀ'eÅ'= H€€€çÇèN$€†"€‰€€‚ÿWe will not discuss genetic algorithms further, in order to discourage you from considering them for your applications. However, pointers to implementations of genetic algorithms are provided in Section if you really insist on playing with them.)(Ä'ŽÅ'& €€€‚ÿÀ¿eÅ'NÇ' Ё€€‚çÛg_ɀ†"€ ‰€çU9ۀ†"€ ‰€çÙg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æíÅÛ/€†"€‰€æäÅÛ/€†"€‰€æåÅÛ/€†"€‰€€çÛg_ɀ‰€€çU9ۀ‰€€çÙg_ɀ‰€€‚‚ÿ Next: War Story: Annealing Arrays Up: Heuristic Methods Previous: Neural Networks AlgorithmsMon Jun 2 23:33:50 EDT 1997)ŽÅ'wÇ'& €€€‚ÿµfNÇ',È'O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1wÇ']È'1UÿÿÿÿÿÿÿÿÍÿÿÿÿ]È'È'$,È'È'" €€€ ÿL]È'ÍÈ'1 (ÿÿÿÿÿÿÿÿÎÍÈ'É'þN(War Story: Annealing ArraysBÈ'É'# €>€€‚ÿWar Story: Annealing Arrays ÍÈ'¬Ì' ‚€€çÜg_ɀ†"€ ‰€ç÷7ۀ†"€ ‰€çÚg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€ææÅÛ/€†"€‰€æçÅÛ/€†"€‰€æèÅÛ/€†"€‰€€çÜg_ɀ‰€€ç÷7ۀ‰€€çÚg_ɀ‰€‚€‚€‚çÍ8ˆ€†"€‰€‚ÿ Next: Parallel Algorithms Up: Combinatorial Search and Heuristic Previous: Genetic AlgorithmsWar Story: Annealing Arrays The war story of Section reported how we used advanced data structures to simulate a new method for sequencing DNA. Our method, interactive sequencing by hybridization (SBH), involved building arrays of specific oligonucleotides on demand. Although the simulation results were very promising to us, most biologists we encountered were suspicious. They needed to see our technique proven in the lab before they would take it seriously.    ©NÉ'a([ „€¡€€ç"˜K€‰€ç -Ðl€†"€‰€†"€Ú€€€€‚ÿBut we got lucky. A biochemist at Oxford University, got interested in our technique, and moreover he had in his laboratory the equipment we needed to test it out. The Southern Array Maker [Sou96], manufactured by Beckman Instruments, could prepare discrete oligonucleotide sequences in 64 parallel rows across a polypropylene substrate. The device constructs arrays by appending single characters to each cell along specific rows and columns of arrays. Figure shows how to construct an array of all purine (A or G) 4-mers by building the prefixes along rows and the suffixes along columns. This technology provided an ideal environment for testing the feasibility of interactive SBH in a laboratory, because with proper programming it gave an¬Ì'a(È' inexpensive way to fabricate a wide variety of oligonucleotide arrays on demand.  ˜I¬Ì'ù(O l€•€€†"€Û€€‚€€€€€€€€‚ÿ  Figure: A prefix-suffix array of all purine 4-mers.  But we had to provide the proper programming. Fabricating complicated arrays requires solving a difficult combinatorial problem. We were given as input a set S of n strings (representing oligonucleotides) to fabricate an array (where m=64 on the Southern apparatus). We had to produce a schedule of row and column commands to realize the set of strings S. We proved that the problem of designing dense arrays was NP-complete, but that didn't really matter. My student Ricky Bradley and I had to solve it anyway.+a($(' € €€‚‚ÿ``If it's hard, it's hard. We are going to have to use a heuristic,'' I told him. ``So how do we model this problem?''``Well, for each string we can identify the possible prefix and suffix pairs that will realize it. For example, the string `ACC' can be realized in four different ways: prefix `' and suffix `ACC', prefix `A' and suffix `CC', prefix `AC' and suffix `C', or prefix `ACC' and suffix `'. We seek the smallest set of prefixes and suffixes that together realize all the given strings,'' Ricky said.”eù(ž(/ ,€Ë€€‚‚€€‚ÿ``Good. This gives us a natural representation for simulated annealing. The state space will consist of all possible subsets of prefixes and suffixes. The natural transitions between states might include inserting or deleting strings from our subsets, or swapping a pair in or out.''``What's a good cost function?'' he asked.``Well, we need as small an array as possible that covers all the strings. How about something like the maximum of number of rows (prefixes) or columns (suffixes) used in our array, plus the number of strings from S that are not yet covered. Try it and let's see what happens.''Y4$( (% €i€€‚ÿRicky went off and implemented a simulated annealing program along these lines. Printing out the state of the solution each time a transition was accepted, it was fun to watch. Starting from a random solution, the program quickly kicked out unnecessary prefixes and suffixes, and the array began shrinking rapidly in size. But after several hundred iterations, progress started to slow. A transition would knock out an unnecessary suffix, wait a while, then add a different suffix back again. After a few thousand iterations, no real improvement was happening.@ž(Q () €/€€‚‚‚ÿ``The program doesn't seem to recognize when it is making progress. The evaluation function only gives credit for minimizing the larger of the two dimensions. Why not add a term to give some credit to the other dimension.''Ricky changed the evaluation function, and we tried again. This time, the program did not hesitate to improve the shorter dimension. Indeed, our arrays started to be skinny rectangles instead of squares.``OK. Let's add another term to the evaluation function to give it points for being roughly square.''®… (ÿ() € €€‚‚‚ÿRicky tried again. Now the arrays were the right shape, and progress was in the right direction. But the progress was slow.``Too many of the prefix/suffix insertion moves don't really seem to affect many strings. Maybe we should skew the random selections so that the important prefix/suffixes get picked more often.''Ricky tried again, Now it converged faster, but sometimes it still got stuck. We changed the cooling schedule. It did better, but was it doing well? Without a lower bound knowing how close we were to optimal, it couldn't really tell how good our solution was. We tweaked and tweaked until our program stopped improving.|XQ ({($ €°€€‚ÿOur final solution refined the initial array by applying the following random moves:³Rÿ(:B(a €¥€Ȁ€€€‚€€€‚€€€‚€€€‚€€€‚€€€‚ÿu swap - swap a prefix/suffix o{(:B(È'n the array with one that isn't.u add - add a random prefix/suffix to the array.u delete - delete a random prefix/suffix from the array.u useful add - add the prefix/suffix with the highest usefulness to the array.u useful delete - delete the prefix/suffix with the lowest usefulness from the array.u string add - randomly select a string not on the array, and add the most useful prefix and/or suffix that covers this string (additional preference is given to a prefix/suffix whose corresponding suffix/prefix is already on the array).Ï{(XF(O l€£€€‚†"€Ü‚€€€€†"€Ý€€‚‚ÿA standard annealing schedule was used, with an exponentially decreasing temperature (dependent upon the problem size) and a temperature-dependent Boltzmann criterion for accepting states that have higher costs. Our final cost function was defined aswhere max is the size of the maximum chip dimension, min is the size of the minimum chip dimension, , and is the number of strings in S currently on the chip.Careful analysis of successful moves over the course of the annealing process suggested a second phase of annealing to speed convergence. Once the temperature reaches a predetermined cutoff point, the temperature schedule was changed to force the temperature to decrease more rapidly, and the probability distribution for choosing moves was altered to include only swap, add, and delete, with preference given to swap moves. This modification sped up late convergence, which had been slower than it was in the early stages of the annealing process.ÉG:B(!J(‚ Ҁ™€€‚†"€Þ€‚€‚çŸ;C€‰€ç -Ðl€†"€‰€ç -Ðl€†"€‰€†"€ß†"€à‚ÿ  Figure: Compression of the HIV array by simulated annealing - after 0, 500, 1,000, and 5,750 iterations  How well did we do? As reported in our paper [BS97], Figure  shows the convergence of a custom array consisting of the 5,716 unique 7-mers of the HIV-virus. Figure  shows snapshots of the state of the chip at four points during the annealing process (0, 500, 1,000, and the final chip at 5,750 iterations). Black pixels represent the first occurrence of an HIV 7-mer, while white pixels represent either duplicated HIV 7-mers or strings not in the HIV input set. The final chip size here is , quite an improvement over the initial size of . It took about fifteen minutes' worth of computation on a desktop workstation to complete the optimization, which was perfectly acceptable for the application.   âXF(+L(( €Å€€€‚ÿBut how well did we do? Since simulated annealing is only a heuristic, we really don't know how close to optimal our solution is. I think we did pretty well, but I can't really be sure. In conclusion, simulated annealing can be the right way to handle complex optimization problems. However, to get the best results, expect to spend more time tweaking and refining your program than you did in writing it in the first place. This is dirty work, but sometimes you have to do it.)!J(TL(& €€€‚ÿÌË+L( N( Ё§€€‚çÜg_ɀ†"€ ‰€ç÷7ۀ†"€ ‰€çÚg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æéÅÛ/€†"€‰€æêÅÛ/€†"€‰€æëÅÛ/€†"€‰€€çÜg_ɀ‰€€ç÷7ۀ‰€€çÚg_ɀ‰€€‚‚ÿ Next: Parallel Algorithms Up: Combinatorial Search and Heuristic Previous: Genetic Algorithms AlgorithmsMon Jun 2 23:33:50 EDT 1997)TL(IN(& €€€‚ÿµf N(þN(O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1IN(/O(1UÿÿÿÿÿÿÿÿÏÿÿÿÿ/O(SO($þN(SO(" €€€ ÿD/O(—O(1¿ÿÿÿÿÿÿÿÿЗO(ÑO(·Å(Parallel Algorithms:SO(ÑO(# €.€€‚ÿParallel Algorithms C2—O( ƒ( ðu€€çÝg_ɀ†"€ÑO( ƒ(SO( ‰€ç÷7ۀ†"€ ‰€çÛg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æìÅÛ/€†"€‰€æÆÛ/€†"€‰€æÆÛ/€†"€‰€€çÝg_ɀ‰€€ç÷7ۀ‰€€çÛg_ɀ‰€‚€‚€€€€€‚ÿ Next: War Story: Going Nowhere Up: Combinatorial Search and Heuristic Previous: War Story: Annealing ArraysParallel AlgorithmsTwo heads are better than one, and more generally, n heads are better than n-1. In our era of computing plenty, parallel processing seems like an exciting technique for solving combinatorial optimization problems. Today many facilities contain networks of workstations, most of them idle at night and underutilized during the day. Why not put them to work?  2 ÑO(R…(% €€€‚ÿParallelism seems like the easy way out of hard problems. Indeed, sometimes, for some problems, parallel algorithms are the most effective solution. High-resolution, real-time graphics applications must render thirty frames per second for realistic animation. Assigning each frame to a distinct processor, or dividing each image into regions assigned to different processors might be the only way to get the job done in time. Large systems of linear equations for scientific applications are routinely solved in parallel.Œh ƒ(ޅ($ €Ð€€‚ÿHowever, there are several pitfalls associated with parallel algorithms that one should be aware of:™jR…(wˆ(/ ,€Õ€Ȁ€€€‚ÿu There is often a small upper bound on the potential win - Suppose that you have access to twenty workstations that can be devoted exclusively to your job. Potentially, these could be used to speed up the fastest sequential program by up to a factor of twenty. That is nice, but much greater performance gains are potentially possible by finding a better sequential algorithm and implementing that. Your time spent parallelizing a code might well be better spent enhancing the sequential version. Performance-tuning tools such as profilers are better developed for sequential machines than for parallel models.bޅ(‹(+ $€Å€Ȁ€€‚ÿu Speedup means nothing - Suppose my parallel program runs 16 times faster on a 16-processor machine then it does on one processor. That's great, isn't it? If you always get linear speedup and have an arbitrary number of processors, you will eventually beat any sequential algorithm. However, a carefully designed sequential algorithm can often beat an easily parallelized code running on a typical parallel machine. The one-processor parallel version of your algorithm is likely to be a crummy sequential algorithm, so measuring speedup typically provides an unfair test of the benefits of parallelism.  »•wˆ(¿(& €+€Ȁ‚ÿThe classic example of this occurs in the minimax game-tree search algorithms used in computer chess programs. Brute-force tree search is embarrassingly easy to parallelize; just put each subtree on a different processor. However, a lot of work gets wasted because the same positions get considered on different machines. Moving from brute-force search to the more clever alpha-beta pruning algorithm can easily save over 99.99% of the work, thus dwarfing any benefits of parallel brute-force search. Alpha-beta can be parallelized, but not easily, and speedups are typically limited to a factor of six or so regardless of how many processors you have.     N‹(JÀ(1 0€€Ȁ€€€€‚ÿu Parallel algorithms are tough to debug - Unless your problem can be decomposed into several independent jobs, the different processors will have to communicate with each other in order to end up with the correct final result. Unfortunately, the non-deterministic nature of this communication makes parallel programs notoriously difficult to debug. Perhaps the best example is Deep Blue, the world-champion chess computer. Although it beat Kasparov, over the years it has lost several games in embarrassing fashion due to ¿(JÀ(SO(bugs, mostly associated with its extensive parallelism.     Œd¿(ÖÂ(( €É€€€‚ÿI recommend considering parallel processing only after repeated attempts at solving the problem sequentially prove too slow. Even then, I would restrict attention to algorithms that parallelize the problem by partitioning the input into distinct tasks, where no communication is needed between the processors, except in collecting the final results. Such large-grain, naive parallelism can be simple enough to be readily implementable and debuggable, because it really reduces to producing a good sequential implementation. Still, there can be pitfalls in this approach, as discussed in the war story below.)JÀ(ÿÂ(& €€€‚ÿÚÙÖÂ(ÙÄ( ЁÃ€€‚çÝg_ɀ†"€ ‰€ç÷7ۀ†"€ ‰€çÛg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÆÛ/€†"€‰€æÆÛ/€†"€‰€æÆÛ/€†"€‰€€çÝg_ɀ‰€€ç÷7ۀ‰€€çÛg_ɀ‰€€‚‚ÿ Next: War Story: Going Nowhere Up: Combinatorial Search and Heuristic Previous: War Story: Annealing Arrays AlgorithmsMon Jun 2 23:33:50 EDT 1997)ÿÂ(Å(& €€€‚ÿµfÙÄ(·Å(O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Å(èÅ(1UÿÿÿÿÿÿÿÿÑÿÿÿÿèÅ( Æ($·Å( Æ(" €€€ ÿNèÅ(ZÆ(1ZÿÿÿÿÿÿÿÿÒZÆ(žÆ(ã)War Story: Going Nowhere FastD! Æ(žÆ(# €B€€‚ÿWar Story: Going Nowhere Fast ZÆ(ºÉ( ‚€€çÞg_ɀ†"€ ‰€ç÷7ۀ†"€ ‰€çÜg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÆÛ/€†"€‰€æÆÛ/€†"€‰€æÆÛ/€†"€‰€€çÞg_ɀ‰€€ç÷7ۀ‰€€çÜg_ɀ‰€‚€‚€‚çlªù(€†"€‰€‚ÿ Next: Exercises Up: Combinatorial Search and Heuristic Previous: Parallel AlgorithmsWar Story: Going Nowhere Fast In Section , I related our efforts to build a fast program to test Waring's conjecture for pyramidal numbers. At that point, my code was fast enough that it could complete the job in a few weeks running in the background on a desktop workstation. This option did not appeal to my supercomputing colleague, however.   ]6žÆ(Ì(' €m€€‚‚ÿ``Why don't we do it in parallel?'' he suggested. ``After all, you have an outer loop doing the same type of calculation on each integer from 1 to 1,000,000,000. I can split this range of numbers into different intervals and run each one of these on a different processor. Watch, it will be easy.''He set to work trying to do our computations on an Intel IPSC-860 hypercube using 32 nodes, with 16 megabytes of memory per node. However, instead of getting answers, over the next few weeks I was treated to a regular stream of e-mail about system reliability:  ’XºÉ(©Í(: B€±€Ȁ€€‚€€‚€€‚ÿu ``Our code is running fine, except one processor died last night. I will rerun.''u ``This time the machine was rebooted by accident, so our long-standing job was killed.''u ``We have another problem. The policy on using our machine is that nobody can command the entire machine for more than thirteen hours, under any condition.''>Ì( )% €3€€‚ÿStill, eventually, he rose to the challenge. Waiting until the machine was stable, he locked out 16 processors (half the computer), divided the integers from 1 to 1,000,000,000 into 16 equal-sized intervals, and ran each interval on its own processor. He spent the next day fending off angry users who couldn't get their work done because of our rogue job. The instant the first processor completed analyzing the numbers from 1 to 62,500,000, he announced to all the people yelling at him that the other processors would soon follow.©Í( ) Æ(R-©Í(^)% €[€€‚ÿBut they didn't. He failed to realize that the time to test each integer increased as the numbers got larger. After all, it would take longer to test whether 1,000,000,000 could be expressed as the sum of three pyramidal number than it would for 100. Thus at slower and slower intervals, each new processor would announce its completion. Because of the architecture of the hypercube, he couldn't return any of the processors until our entire job was completed. Eventually, half the machine and most of its users were held hostage by one, final interval.»| ))? L€û€€‚çQr€†"€‰€€‚ÿWhen the job finally completed, the numbers were passed on to the Nobel Prize winner who had requested them. It turns out he had been curious about the problem because his father had made the conjecture back in 1928. There had never been a more important scientific reason to justify the computation in the first place. Indeed, no results from the computation ever appeared in print.  What conclusions can be drawn from this? Before devoting heroic efforts to solve a problem efficiently, make sure that it really needs to be solved, and solved quickly. If you are going to parallelize a problem, be sure to balance the load carefully among the processors. Proper load balancing, using either back-of-the-envelope calculations or the partition algorithm of Section , would have significantly reduced the time we needed the machine, and his exposure to the wrath of his colleagues.)^)B)& €€€‚ÿÃÂ)) Ё•€€‚çÞg_ɀ†"€ ‰€ç÷7ۀ†"€ ‰€çÜg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÆÛ/€†"€‰€æÆÛ/€†"€‰€æCÆÛ/€†"€‰€€çÞg_ɀ‰€€ç÷7ۀ‰€€çÜg_ɀ‰€€‚‚ÿ Next: Exercises Up: Combinatorial Search and Heuristic Previous: Parallel Algorithms AlgorithmsMon Jun 2 23:33:50 EDT 1997)B).)& €€€‚ÿµf)ã)O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1.) )1UÿÿÿÿÿÿÿÿÓÿÿÿÿ )8 )$ã)8 )" €€€ ÿ:  )r )1LÿÿÿÿÿÿÿÿÔr )¢ )VG)Exercises0 8 )¢ )# €€€‚ÿExercises Âœr )d ) ؁‹€€çen_ɀ†"€ ‰€ç÷7ۀ†"€ ‰€çÝg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ:ÆÛ/€†"€‰€æ;ÆÛ/€†"€‰€æ<ÆÛ/€†"€‰€€çen_ɀ‰€€ç÷7ۀ‰€€çÝg_ɀ‰€‚€‚€‚ÿ Next: Implementation Challenges Up: Combinatorial Search and Heuristic Previous: War Story: Going NowhereExercises ·¢ ))€ ;€Ȁ€€€€†"€w†"€á†"€ç€€‚€€€€€€†"€â†"€â†"€ã†"€ä†"€å†"€æ†"€ç‚ÿ1. (*) A derangement is a permutation p of such that no item is in its proper position, i.e. for all .   Write an efficient backtracking program with pruning that constructs all the derangements of n items.2. (*) Multisets are allowed to have repeated elements. A multiset of n items may thus have fewer than n! distinct permutations. For example, has only six different permutations: , , , , , and .   Design and implement an efficient algorithm for constructing all permutations of a multiset.¡Bd )È@)_ Œ€‡€Ȁçó‘¶ü€†"€‰€‚€€€€€€€€€€€€‚ÿ3. (*) Design and implement an algorithm for testing whether two graphs are isomorphic to each other. The graph isomorphism problem is discussed in Section . With proper pruning, graphs on hundreds of vertices can be tested reliably.4. (**) Design and implement an algorithm for solving the subgraph isomorphism problem. Given graphs G and H, does there exist a subgraph H' of )È@)8 )H such that G is isomorphic to H'. How does your program perform on such special cases of subgraph isomorphism as Hamiltonian cycle, clique, independent set, and graph isomorphism.ƒ3)KD)P n€k€Ȁ繿 耆"€‰€‚€€€€†"€‚ÿ5. (*) Design and implement an algorithm for solving the set cover problem, discussed in Section . Use it to solve special-case vertex cover problems as well as general set cover problems.6. (**) In the turnpike reconstruction problem, you are given n(n-1)/2 distances in sorted order. The problem is to find the positions of the points on the line that give rise to these distances. For example, the distances can be determined by placing the second point 1 unit from the first, the third point 3 from the second, and the fourth point 2 from the third. Design and implement an efficient algorithm to report all solutions to the turnpike reconstruction problem. Exploit additive constraints when possible to minimize search. With proper pruning, problems with hundreds of points can be solved reliably.  ,È@)wD)' € €€€‚ÿ)KD) D)& €€€‚ÿØ×wD)xF) Ё¿€€‚çen_ɀ†"€ ‰€ç÷7ۀ†"€ ‰€çÝg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ=ÆÛ/€†"€‰€æ>ÆÛ/€†"€‰€æ?ÆÛ/€†"€‰€€çen_ɀ‰€€ç÷7ۀ‰€€çÝg_ɀ‰€€‚‚ÿ Next: Implementation Challenges Up: Combinatorial Search and Heuristic Previous: War Story: Going Nowhere AlgorithmsMon Jun 2 23:33:50 EDT 1997) D)¡F)& €€€‚ÿµfxF)VG)O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¡F)‡G)1UÿÿÿÿÿÿÿÿÕÿÿÿÿ‡G)«G)$VG)«G)" €€€ ÿH‡G)óG)1cÿÿÿÿÿÿÿÿÖóG)1H))Ž)Problems and Reductions>«G)1H)# €6€€‚ÿProblems and Reductions -(óG)^K) ؁a€€çSf_ɀ†"€ ‰€çø7ۀ†"€ ‰€çø7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ@ÆÛ/€†"€‰€æAÆÛ/€†"€‰€æBÆÛ/€†"€‰€€çSf_ɀ‰€€çø7ۀ‰€€çø7ۀ‰€‚€‚€‚ÿ Next: Simple Reductions Up: Intractable Problems and Approximations Previous: Intractable Problems and ApproximationsProblems and ReductionsThroughout this book we have encountered problems, such as the traveling salesman problem, for which we couldn't find any efficient algorithm. By the early 1970s, literally hundreds of problems were stuck in this swamp. The theory of NP-completeness provided the tools needed to show that all of these problems were really the same thing.â±1H)@L)1 0€c€€€€€€‚ÿThe key idea behind demonstrating the hardness of a problem is that of a reduction. Suppose that I gave you the following algorithm to solve the Bandersnatch problem:   €^K)\O)x Ÿ€I€€‚€€‚€€€€‚€€‚€€€€‚€€€€€€€€€‚ÿ Bandersnatch(G) Translate the input G to an instance of the Bo-billy problem Y. Call the subroutine Bo-billy on Y to solve this instance. Return the answer of Bo-billy(Y) as the answer to Bandersnatch(G). It is important to see that this algorithm correctly solves the Bandersnatch problem provided that the translation to Bo-billy always preserves the correctness of the answer. In other words, the translation has the property that for any instance of G, Bandersnatch(G) = Bo-billy(Y). A translation of instances from one type of problem to instances of another type such that the answers are preserved is called a reduction.²p@L)€)B T€à€€€€€€€€€€€€‚ÿNow suppose this reduction translates G to Y in O(P(n)) time. There are two possible i\O)€)«G)mplications:„\O)ƒ) Ѐ €Ȁ€€€€€€€€€€€€€€€€€€€‚€€€†"€è†"€é€€‚ÿu If my Bo-billy subroutine ran in O(P'(n)), this means I could solve the Bandersnatch problem in O(P(n)+P'(n)) by spending the time to translate the problem and then the time to execute the Bo-Billy subroutine.u If I know that is a lower bound on computing Bandersnatch, meaning there definitely exists no faster way to solve it, then must be a lower bound to compute Bo-billy. Why? If I could solve Bo-billy any faster, then I could solve Bandersnatch in faster time by using the above simulation, thus violating my lower bound. This implies that there can be no way to solve Bo-billy any faster than claimed.Ò€)'†)6 :€¥€€‚€€€€€‚ÿThis second argument is the approach that we will use to prove problems hard. Essentially, this reduction shows that Bo-billy is at least as hard as Bandersnatch, and therefore once we believe that Bandersnatch is hard, we have a tool for proving other problems hard.Reductions, then, are operations that convert one problem into another. To describe them, we must be somewhat rigorous in our definition of a problem. A problem is a general question, with parameters for the input and conditions on what constitutes a satisfactory answer or solution. An instance is a problem with the input parameters specified. The difference can be made clear by an example. The traveling salesman problem is defined thus:   ˜Cƒ)¿ˆ)U x€‹€€€€€€‚€†"€ê†"€ë‚‚€€€‚ÿInput: A weighted graph G. Output: Which tour minimizes ?Thus any weighted graph defines an instance of TSP. Each particular instance has at least one minimum cost tour. The general traveling salesman problem asks for an algorithm to find the optimal tour for all possible instances.Any problem with answers restricted to yes and no is called a decision problem. Most interesting optimization problems can be phrased as decision problems that capture the essence of the computation. For example, the traveling salesman decision problem can be defined thus: /'†)>‹)P n€a€€€€€€€€‚€†"€ì€€‚€‚ÿInput: A weighted graph G and integer k. Output: Does there exist a TSP tour with cost ? It should be clear that the decision version captures the heart of the traveling salesman problem, for if you had a program that gave fast solutions to the decision problem, you could do a binary search with different values of k to quickly hone in on the correct solution.Therefore, from now on we will talk only about decision problems, because it is easier to reduce one problem to another when the only possible answers to both are true or false.)¿ˆ)g‹)& €€€‚ÿäã>‹)K) Ё×€€‚çSf_ɀ†"€ ‰€çø7ۀ†"€ ‰€çø7ۀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€ænÆÛ/€†"€‰€æeÆÛ/€†"€‰€æfÆÛ/€†"€‰€€çSf_ɀ‰€€çø7ۀ‰€€çø7ۀ‰€€‚‚ÿ Next: Simple Reductions Up: Intractable Problems and Approximations Previous: Intractable Problems and Approximations AlgorithmsMon Jun 2 23:33:50 EDT 1997)g‹)t)& €€€‚ÿµfK))Ž)O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1t)ZŽ)1Uÿÿÿÿÿÿÿÿ×ÿÿÿÿZŽ)~Ž)$)Ž)~Ž)" €€€ ÿBZŽ)ÀŽ)1ÂÿÿÿÿÿÿÿÿØÀŽ)øŽ)8Ä)Simple Reductions8~Ž)øŽ)# €*€€‚ÿSimple Reductions ®ŠÀŽ)²Á) ށ]€€çTf_ɀ†"€ ‰€çø7ۀ†"€ ‰€ç\f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€ægÆÛ/€†"€‰€æhÆÛ/€†"€‰€æiÆÛ/€†"€‰€€çTf_ɀ‰€€çø7ۀ‰€€ç\f_ɀ‰€‚€‚€€‚ÿøŽ)²Á)~Ž) Next: Hamiltonian Cycles Up: Intractable Problems and Approximations Previous: Problems and ReductionsSimple ReductionsSince they can be used either to prove hardness or to give efficient algorithms, reductions are powerful tools for the algorithm designer to be familiar with. The best way to understand reductions is to look at some simple ones.)øŽ)ÛÁ)& €€€‚ÿ&²Á)Â)# €€€‚ÿ·gÛÁ)žÂ)P p€Î€Ȁçç¶ÌI€‰€‚€çHEÓš€‰€‚€ç©ÓÙ€‰€‚ÿu Hamiltonian Cycles u Independent Set and Vertex Cover u Clique and Independent Set&Â)ÞÂ)# €€€‚ÿ)žÂ)Ã)& €€€‚ÿS+ÞÂ)ZÃ)( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)Ã)ƒÃ)& €€€‚ÿµfZÃ)8Ä)O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ƒÃ)iÄ)1UÿÿÿÿÿÿÿÿÙÿÿÿÿiÄ)Ä)$8Ä)Ä)" €€€ ÿCiÄ)ÐÄ)1dÿÿÿÿÿÿÿÿÚÐÄ) Å)&*Hamiltonian Cycles9Ä) Å)# €,€€‚ÿHamiltonian Cycles ÊÐÄ)ÌÇ) ‚_€€çUf_ɀ†"€ ‰€çSf_ɀ†"€ ‰€çSf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æjÆÛ/€†"€‰€ækÆÛ/€†"€‰€ælÆÛ/€†"€‰€€çUf_ɀ‰€€çSf_ɀ‰€€çSf_ɀ‰€‚€ ‚€ç$n‚Š€†"€‰€€‚ÿ Next: Independent Set and Vertex Up: Simple Reductions Previous: Simple ReductionsHamiltonian CyclesThe Hamiltonian cycle problem is one of the most famous in graph theory. It seeks a tour that visits each vertex of a given graph exactly once. It has a long history and many applications, as discussed in Section . Formally, it is defined:   Y Å)%Ê)< F€;€€€€€€‚€€€‚ÿInput: An unweighted graph G. Output: Does there exist a simple tour that visits each vertex of G without repetition? Hamiltonian cycle has some obvious similarity to the traveling salesman problem. Both problems ask for a tour to visit each vertex exactly once. There are also differences between the two problems. TSP works on weighted graphs, while Hamiltonian cycle works on unweighted graphs. The following reduction from Hamiltonian cycle to traveling salesman shows that the similarities are greater than the differences:ƯÌÇ)ëÎ) üc€€‚€€€€€€‚€€€€€€€€€€‚€€€€‚€€€€‚€€€€†"€í€€€€€€€€€€€€‚€€€€‚€€€†"€î€€€€€€€€€€€€€€€€€€‚ÿ HamiltonianCycle(G=(V,E)) Construct a complete weighted graph G'=(V',E') where V'=V. n = |V| for i = 1 to n do for j = 1 to n do if then w(i,j) = 1 else w(i,j) = 2 Return the answer to Traveling-Salesman(G',n). The actual reduction is quite simple, with the translation from unweighted to weighted graph easily performed in linear time. Further, this translation is designed to ensure that the answers of the two problems will be identical. If the graph G has a Hamiltonian cycle , then this exact same tour will correspond to n edges in E', each with weight 1. Therefore, this gives a TSP tour of G' of weight exactly n. If G does not have a Hamiltonian cycle, then there can be no such TSP tour in G', because the only way to get a tour of cost n in G would be to use only edges of weight 1, which implies a Hamiltonian cycle in G.g?%Ê)^*( €€€€‚ÿThis reduction is both efficient and truth preserving. A fast algorithm for TSP would imply a fast algorithm for Hamiltonian cycle, while a hardness proof for Hamiltonian cycle would imply that TSP is hard. Since the latter is the caseëÎ)^*Ä), this reduction shows that TSP is hard, at least as hard as Hamiltonian cycle.)ëÎ)‡*& €€€‚ÿÁÀ^*H* Ё‘€€‚çUf_ɀ†"€ ‰€çSf_ɀ†"€ ‰€çSf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æmÆÛ/€†"€‰€æ™ÆÛ/€†"€‰€æÆÛ/€†"€‰€€çUf_ɀ‰€€çSf_ɀ‰€€çSf_ɀ‰€€‚‚ÿ Next: Independent Set and Vertex Up: Simple Reductions Previous: Simple Reductions AlgorithmsMon Jun 2 23:33:50 EDT 1997)‡*q*& €€€‚ÿµfH*&*O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1q*W*1UÿÿÿÿÿÿÿÿÛÿÿÿÿW*{*$&*{*" €€€ ÿQ W*Ì*1çÿÿÿÿÿÿÿÿÜÌ**ÕB*Independent Set and Vertex CoverG${**# €H€€‚ÿIndependent Set and Vertex Cover |]Ì** ‚Í€€çVf_ɀ†"€ ‰€çSf_ɀ†"€ ‰€çTf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ‘ÆÛ/€†"€‰€æ’ÆÛ/€†"€‰€æ“ÆÛ/€†"€‰€€çVf_ɀ‰€€çSf_ɀ‰€€çTf_ɀ‰€‚€ ‚€‚çbQù†"€‰€€‚ÿ Next: Clique and Independent Set Up: Simple Reductions Previous: Hamiltonian CyclesIndependent Set and Vertex Cover The vertex cover problem, discussed more thoroughly in Section , asks for a small set of vertices that contacts each edge in a graph. More formally:   ™*( *• ø€€€€€€€€€€†"€ï€‚€€€€€†"€ð€€‚†"€ñ€€‚€€çBEÓš€†"€‰€‚ÿInput: A graph G=(V, E) and integer . Output: Is there a subset S of at most k vertices such that every has at least one vertex in S?  Figure: Circled vertices form a vertex cover, the dark vertices an independent set  It is trivial to find a vertex cover of a graph, for the cover can consist of all of the vertices. More tricky is to cover the edges using as small a set of vertices as possible. For the graph of Figure , four of the eight vertices are sufficient to cover.*- *„ ր €€€€€€€€€€€€†"€ò†"€óçÃnm€†"€‰€€‚€€€†"€ï€‚ÿA set of vertices S of graph G is independent if there are no edges (x,y) where and , meaning there are no edges between any two vertices in the independent set. As discussed in Section , the independent set problem arises in facility location problems. The maximum independent set decision problem is formally defined:   Input: A graph G and integer . Dò( *q *R r€å€€€€€€€€€€€€€€€€€‚ÿOutput: Does there exist an independent set of k vertices in G? Both vertex cover and independent set are problems that revolve around finding special subsets of vertices, the first with representatives of every edge, the second with no edges. If S is the vertex cover of G, the remaining vertices S-V must form an independent set, for if there were an edge with both vertices in S-V, then S could not have been a vertex cover. This gives us a reduction between the two problems:- * @*u ž€5€€‚€€€€‚€€€€‚€€€€€€‚€€€€‚€€‚ÿ VertexCover(G,k) G' = G k' = |V| - k Return the answer to IndependentSet(G',k') Again, a simple reduction shows that the problems are identical. Notice how this translation occurs without any knowledge of the answer. We transform the input, not the solution. This reduction shows that the hardness of vertex cover imples that independent set must also be hard. It is easy to reverse the roles of the two problems in this reduction, thus proving that both of these problems are equally hard.q * @*{*)q *5@*& €€€‚ÿÂÁ @*÷A* Ё“€€‚çVf_ɀ†"€ ‰€çSf_ɀ†"€ ‰€çTf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ”ÆÛ/€†"€‰€æ•ÆÛ/€†"€‰€æ–ÆÛ/€†"€‰€€çVf_ɀ‰€€çSf_ɀ‰€€çTf_ɀ‰€€‚‚ÿ Next: Clique and Independent Set Up: Simple Reductions Previous: Hamiltonian Cycles AlgorithmsMon Jun 2 23:33:50 EDT 1997)5@* B*& €€€‚ÿµf÷A*ÕB*O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 B*C*1UÿÿÿÿÿÿÿÿÝÿÿÿÿC**C*$ÕB**C*" €€€ ÿKC*uC*1% ÿÿÿÿÿÿÿÿÞuC*¶C*ªM*Clique and Independent SetA*C*¶C*# €<€€‚ÿClique and Independent Set öuC*ÈE* ‚ÿ€€çWf_ɀ†"€ ‰€çSf_ɀ†"€ ‰€çUf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ—ÆÛ/€†"€‰€æ˜ÆÛ/€†"€‰€æÄÆÛ/€†"€‰€€çWf_ɀ‰€€çSf_ɀ‰€€çUf_ɀ‰€‚€ ‚€‚ç 4h€†"€‰€‚ÿ Next: Satisfiability Up: Simple Reductions Previous: Independent Set and VertexClique and Independent Set Consider the clique problem, further discussed in Section :L¡¶C*I*« $M€€†"€ô€€€‚€€€€€€€†"€õ€‚€€€†"€ö†"€÷€€€€ç¢ÓÙ€†"€‰€€€€€€€‚ÿ  Figure: A small graph with a five-vertex clique   Input: A graph G=(V,E) and integer . Output: Does the graph contain a clique of j vertices; i.e. is there a subset , where , such that every pair of vertices in S defines an edge of G? For example, the graph in Figure contains a clique of five vertices. In the independent set problem, we looked for a subset S with no edges between two vertices of S. However, for a clique, we insist that there always be an edge between two vertices. A reduction between these problems results by reversing the roles of edges and non-edges, an operation known as complementing the graph:  <©ÈE*PL*“ ô€S€€‚€€‚€€€€€€€€€€‚€€€€€€€€€€€€‚€€€€‚€€‚ÿ IndependentSet(G,k) Construct a graph G=(V',E') where V'=V, and For all (i,j) not in E, add (i,j) to E' Return the answer to Clique(G',k) These last two reductions provide a chain linking three different problems. The hardness of clique is implied by the hardness of independent set, which is implied by the hardness of vertex cover. By constructing reductions in a chain, we link together pairs of problems in implications of hardness. Our work is done as soon as all these chains begin with a single problem that is accepted as hard. Satisfiability is the problem that serves as the first link in this chain.)I*yL*& €€€‚ÿS+PL*ÌL*( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)yL*õL*& €€€‚ÿµfÌL*ªM*O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1õL*ÛM*1UÿÿÿÿÿÿÿÿßÿÿÿÿÛM*ÿM*$ªM*ÿM*" €€€ ÿ?ÛM*>N*1£ ÿÿÿÿÿÿÿÿà>N*sN*èˆ*Satisfiability5ÿM*sN*# €$€€‚ÿSatisfiability Æž>N*E* ꁁ€€çXf_ɀ†"€ ‰€çø7ۀ†"€ ‰€çVf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ»ÆÛ/€†"€‰€æŒÆÛ/€†"€‰€æœÆÛ/€†"€‰€€çXf_ɀ‰€€çø7ۀ‰€€çVf_ɀ‰€‚€‚€€€€‚ÿ Next: The Theory of NP-Completeness Up: Intractable Problems and AppsN*E*ÿM*roximations Previous: Clique and Independent SetSatisfiabilityTo prove the hardness of different problems using reductions, we need to start with a single problem that is absolutely, certifiably, undeniably hard. The mother of all NP-complete problems is a logic problem named satisfiability:   J«sN*„*Ÿ i€€€€€€€€€€‚€€€†"€ø†"€ù†"€ú†"€û†"€†"€†"€ü†"€û†"€ý€€‚ÿInput: A set of Boolean variables V and a set of clauses C over V. Output: Does there exist a satisfying truth assignment for C, i.e. a way to set the variables either true or false so that each clause contains at least one true literal? This can be made clearer with two examples. Suppose that over the Boolean variables . We use to denote the complement of the variable , so we would get credit for satisfying a particular clause containing if , or a clause containing if . Therefore, satisfying a particular set of clauses involves making a series of n true or false decisions, trying to find the right truth assignment to satisfy all of them. ›E*š†*p ®€G€€†"€ù†"€þ†"€ÿ†"€†"€m†"€†"€n†"€€‚ÿThis example set of clauses can be satisfied by simply setting or . However, consider the set of clauses . There can be no satisfying assignment because must be in order to satisfy the third clause, which means that must be to satisfy the second clause, which then leaves the first clause unsatisfiable. Although you try, and you try, and you try and you try, you can't get no satisfaction.)„*Æ*& €€€‚ÿ&š†*é†*# €€€‚ÿ?Æ*h‡*@ P€~€Ȁçkðæŀ‰€‚€çÌ~í$€‰€‚ÿu The Theory of NP-Completeness u 3-Satisfiability&é†*Ž‡*# €€€‚ÿ)h‡*·‡*& €€€‚ÿS+Ž‡* ˆ*( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)·‡*3ˆ*& €€€‚ÿµf ˆ*èˆ*O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b13ˆ*‰*1Uÿÿÿÿÿÿÿÿáÿÿÿÿ‰*=‰*$èˆ*=‰*" €€€ ÿN‰*‹‰*1kÿÿÿÿÿÿÿÿ⋉*ω*1É*The Theory of NP-CompletenessD!=‰*ω*# €B€€‚ÿThe Theory of NP-Completeness ‘w‹‰*`Ž* ‚€€çYf_ɀ†"€ ‰€çWf_ɀ†"€ ‰€çWf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æŸÆÛ/€†"€‰€æ¿ÆÛ/€†"€‰€æÀÆÛ/€†"€‰€€çYf_ɀ‰€€çWf_ɀ‰€€çWf_ɀ‰€‚€ ‚€çï'Uý€†"€‰€‚ÿ Next: 3-Satisfiability Up: Satisfiability Previous: SatisfiabilityThe Theory of NP-CompletenessFor a variety of social and technical reasons, it is well accepted that satisfiability is a hard problem, one for which no worst-case polynomial-time algorithm exists. Literally every top-notch algorithm expert in the world (and countless lesser lights) has directly or indirectly tried to come up with a fast algorithm to test whether a given set of clauses is satisfiable, but all have failed. Further, many strange and impossible-to-believe things in the field of computational complexity have been shown to be true if there exists a fast satisfiability algorithm. Satisfiability is a hard problem, and it is important to accept this. See Section for more on the satisfiability problem and its applications.  Žω* Ã*3 4€ €€‚€€€€‚ÿThe theory of NP-completeness rests on a foundation of rigorous but subtle definitions from automata and formal language theory. This terminology is typically confusing to or misused by beginners who lack a mastery of these foundations, and it is not really essential to the practical aspects of designing and applying reductions. For completeness, however, we br`Ž* Ã*=‰*iefly define the key terms below.A problem is said to be polynomial (or in the class P) if it can be solved in time polynomial in its size. A problem is said to be nondeterministically polynomial (or in the class NP) if a conjectured answer can be verified in time polynomial in its size. The traveling salesman decision problem is not known to be in P, because there is no known polynomial-time algorithm for it. However, the problem is in NP, because if we are given a candidate tour, we can efficiently add up the cost of the associated edges and verify whether the total is at most the cost bound k. It is typically straightforward to verify whether the answer to a problem is correct, and it certainly can be no harder than actually finding the answer in the first place.    Y`Ž*yÆ*: B€?€€€€€€€€€‚ÿThrough a complicated proof, it has been established that satisfiability is at least as hard as any problem in NP. This means that if a fast (i.e. polynomial-time) algorithm is discovered for solving satisfiability, this will yield a fast algorithm for every problem in NP. Since essentially every problem mentioned this book is in NP, this would be an enormously powerful and surprising result. We say that a problem is NP-hard if, like satisfiability, it is at least as hard as any problem in NP. We say that a problem is NP-complete if it is NP-hard, and also in NP itself. Because NP is such a large class of problems, most NP-hard problems you encounter will in fact be complete, and the issue can always be settled by giving a (usually simple) verification strategy for the problem.) Ã*¢Æ*& €€€‚ÿ±°yÆ*SÈ* Ёq€€‚çYf_ɀ†"€ ‰€çWf_ɀ†"€ ‰€çWf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÁÆÛ/€†"€‰€æÂÆÛ/€†"€‰€æÃÆÛ/€†"€‰€€çYf_ɀ‰€€çWf_ɀ‰€€çWf_ɀ‰€€‚‚ÿ Next: 3-Satisfiability Up: Satisfiability Previous: Satisfiability AlgorithmsMon Jun 2 23:33:50 EDT 1997)¢Æ*|È*& €€€‚ÿµfSÈ*1É*O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1|È*bÉ*1UÿÿÿÿÿÿÿÿãÿÿÿÿbÉ*†É*$1É*†É*" €€€ ÿDbÉ*ÊÉ*1÷ÿÿÿÿÿÿÿÿäÊÉ*Ê*a+"3-Satisfiability "7†É*Ê*# €(€€‚ÿ3-Satisfiability ªœÊÉ*«Í* êK€€çZf_ɀ†"€ ‰€çWf_ɀ†"€ ‰€çXf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æïÆÛ/€†"€‰€ææÆÛ/€†"€‰€æçÆÛ/€†"€‰€€çZf_ɀ‰€€çWf_ɀ‰€€çXf_ɀ‰€‚€ ‚€†"€‚ÿ Next: Difficult Reductions Up: Satisfiability Previous: The Theory of NP-Completeness3-SatisfiabilitySatisfiability's role as the first NP-complete problem implies that the problem is hard to solve in the worst case, but certain instances of the problem are not necessarily so tough. Suppose that each clause contains exactly one literal. To satisfy such a clause, we have to appropriately set that literal, so we can repeat this argument for every clause in the problem instance. Only when we have two clauses that directly contradict each other, such as , will the set not be satisfiable.  ÷ªÊ*®+M h€U€€€‚€€€€€€€€‚€€€‚ÿSince clause sets with only one literal per clause are easy to satisfy, we are interested in slightly larger classes. Exactly what is the clause size at which the problem turns from polynomial to hard? This transition occurs when each clause contains three literals, the so-called 3-satisfiability problem, or 3-SAT: Input: A collection of clauses C where each clause contains exactly 3 literals, over a set of Boolean variables V. Output: Is there a truth assignment to V such that each clause is sati«Í*®+†É*sfied? Since this is a more restricted problem than satisfiablity, the hardness of 3-SAT implies that satisfiability is hard. The converse isn't true, as the hardness of general satisfiability might depend upon having long clauses. We can show the hardness of 3-SAT using a reduction that translates every instance of satisfiability into an instance of 3-SAT without changing the result of whether it is satisfiable.àŠ«Í*Ž+: B€O€€€€†"€€€‚ÿThis reduction transforms each clause independently based on its length, by adding new Boolean variables along the way. Suppose clause contained k literals:ŽË®++à T³€Ȁ€€€€†"€†"€«€€†"€†"€†"€†"€†"€ †"€‚€€€€†"€ †"€m†"€ †"€ †"€ †"€‚ÿu If k=1, meaning that , we create two new variables and four new 3-literal clauses: , , , . Note that the only way that all four of these clauses can be simultaneously satisfied is if , which must be the case if the original were to be satisfied.u If k=2, meaning that , we create one new variable and two new clauses: , . Again, the only way to satisfy both of these clauses is to have at least one of and be true.Zˎ+v+ 쀧€Ȁ€€€†"€†"€†"€‚€€€€†"€€€€€†"€†"€†"€†"€‚ÿu If k=3, meaning that , we copy into the 3-SAT instance unchanged: .u If k>3, meaning that , create n-3 new variables and n-2 new clauses in a chain, where for , , , and ."¬+˜+v º€g€€†"€†"€†"€†"€†"€†"€†"€€€€€€€‚ÿThe most complicated case is that of the large clauses. If none of the original variables are , then there are not enough additional variables to be able to satisfy all of the new subclauses. You can satisfy by setting , but this forces , and so on until finally cannot be satisfied. But if any single literal , then we have n-3 free variables and n-3 remaining 3-clauses, so we can satisfy each of them.þ˜v+– +f š€5€€€€€€€€€€€€‚†"€çï'Uý€†"€‰€€‚ÿThis transform takes O(m+n) time if there were n clauses and m total literals in the SAT instance. Since any SAT solution also satisfies the 3-SAT instance and any 3-SAT solution sets the variables giving a SAT solution, the transformed problem is equivallent to the original.Note that a slight modification to this construction would serve to prove that 4-SAT, 5-SAT, or any -SAT is also NP-complete. However, this construction breaks down when we try to use it for 2-SAT, since there is no way to stuff anything into the chain of clauses. It turns out that resolution gives a polynomial-time algorithm for 2-SAT, as discussed in Section .)˜+¿ +& €€€‚ÿÄÖ +ƒ + Ё—€€‚çZf_ɀ†"€ ‰€çWf_ɀ†"€ ‰€çXf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æèÆÛ/€†"€‰€æéÆÛ/€†"€‰€æêÆÛ/€†"€‰€€çZf_ɀ‰€€çWf_ɀ‰€€çXf_ɀ‰€€‚‚ÿ Next: Difficult Reductions Up: Satisfiability Previous: The Theory of NP-Completeness AlgorithmsMon Jun 2 23:33:50 EDT 1997)¿ +¬ +& €€€‚ÿµfƒ +a+O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¬ +’+1Uÿÿÿÿÿÿÿÿåÿÿÿÿ’+¶+$a+¶+" €€€ ÿE’+û+1^ÿÿÿÿÿÿÿÿæû+6+dG+Difficult Reductions;¶+6+# €0€€‚ÿDifficult Reductions upû+·B+ ؁ñ€€ç[f_ɀ†"€ ‰€çø7ۀ†"€ ‰€çYf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æëÆÛ/€†"€‰€æìÆÛ/€†"€‰€æíÆÛ/€†"€‰€6+·B+¶+€ç[f_ɀ‰€€çø7ۀ‰€€çYf_ɀ‰€‚€‚€‚ÿ Next: Integer Programming Up: Intractable Problems and Approximations Previous: 3-SatisfiabilityDifficult ReductionsNow that both satisfiability and 3-SAT are known to be hard, we can use either of them in reductions. What follows are a pair of more complicated reductions, designed to serve both as examples for how to proceed and to increase our repertoire of known hard problems from which we can start. Many reductions are quite intricate, because we are essentially programming one problem in the language of a significantly different problem.  m?6+$E+. *€€€€€€‚ÿOne perpetual point of confusion is getting the direction of the reduction right. Recall that we must transform every instance of a known NP-complete problem into an instance of the problem we are interested in. If we perform the reduction the other way, all we get is a slow way to solve the problem of interest, by using a subroutine that takes exponential time. This always is confusing at first, for this direction of reduction seems bass-ackwards. Check to make sure you understand the direction of reduction now, and think back to this whenever you get confused.)·B+ME+& €€€‚ÿ&$E+sE+# €€€‚ÿq1ME+äE+@ P€b€Ȁ玛ú €‚€ç:8€‰€‚ÿu Integer Programming u Vertex Cover&sE+ F+# €€€‚ÿ)äE+3F+& €€€‚ÿS+ F+†F+( €V€€‚€‚ÿAlgorithmsMon Jun 2 23:33:50 EDT 1997)3F+¯F+& €€€‚ÿµf†F+dG+O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¯F+•G+1Uÿÿÿÿÿÿÿÿçÿÿÿÿ•G+¹G+$dG+¹G+" €€€ ÿD•G+ýG+1åÿÿÿÿÿÿÿÿèýG+7H+êÀ+Integer Programming:¹G+7H+# €.€€‚ÿInteger Programming š‹ýG+ßJ+ ‚)€€ç‡f_ɀ†"€ ‰€çZf_ɀ†"€ ‰€çZf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æîÆÛ/€†"€‰€æÇÛ/€†"€‰€æÇÛ/€†"€‰€€ç‡f_ɀ‰€€çZf_ɀ‰€€çZf_ɀ‰€‚€ ‚€ç^ç̀†"€‰€€‚ÿ Next: Vertex Cover Up: Difficult Reductions Previous: Difficult ReductionsInteger ProgrammingAs discussed in Section , integer programming is a fundamental combinatorial optimization problem. It is best thought of as linear programming with the variables restricted to take only integer (instead of real) values.   (–7H+M+’ ò€9€€€€€€€€€€€€€€‚€€€†"€‚†"€‚†"€‚†"€ ‚†"€!†"€"‚ÿInput: A set V of integer variables, a set of inequalities over V, a maximization function f(V), and an integer B. Output: Does there exist an assignment of integers to V such that all inequalities are true and ? Consider the following two examples. SupposeA solution to this would be , . Not all problems have realizable solutions, however. For the following problem:°ßJ+"€+_ Œ€i€€†"€‚†"€‚†"€#‚€€€€†"€$‚‚‚ÿthe maximum value of f(v) is (given the constraints), and so there can be no solution to the associated decision problem.We show that integer programming is hard using a reduction from 3-SAT. For this particular reduction, general satisfiability would work just as well, although usually 3-SAT makes reductions easier.In which direction must the reduction go? We want to prove integer programming that is hard, and we know that 3-SAT is hard. If I could solve 3-SAT using integer programming and integer programming were easy, this would mean that satisfiability would be easy. Now the direction should be clear; we have to translate 3-SAT into M+"€+¹G+integer programming.šxM+ʂ+0 .€ó€€‚†"€‚ÿWhat should the translation look like? Every satisfiability instance contains Boolean (true/false) variables and clauses. Every integer programming instance contains integer variables (values restricted to 0,1,2,...) and constraints. A reasonable idea is to make the integer variables correspond to Boolean variables and have constraints serve the same role as the clauses do in the original problem.Our translated integer programming problem will have twice as many variables as the SAT instance, one for each variable and one for its complement. For each variable in the set problem, we will add the following constraints:—%"€+a„+r ²€Y€Ȁ€€†"€%†"€&†"€'†"€†"€‚€€†"€†"€(‚ÿu To restrict each integer programming variable to values of 0 or 1, we add constraints and . Thus they correspond to values of and .u To ensure that exactly one of the two integer programming variables associated with a given SAT variable is , add constraints so that .׏ʂ+8‡+H ^€%€€†"€†"€)‚†"€*€€‚ÿFor each clause in the 3-SAT instance, construct a constraint: . To satisfy this constraint, at least one the literals per clause must be set to 1, thus corresponding to a true literal. Satisfying this constraint is therefore equivalent to satisfying the clause.The maximization function and bound prove relatively unimportant, since we have already encoded the entire 3-SAT instance. By using and B=0, we ensure that they will not interfere with any variable assignment satisfying all the inequalities. Clearly, this reduction can be done in polynomial time. To establish that this reduction preserves the answer, we must verify two things:©(a„+á‰+ Ѐ]€Ȁ€€€†"€€€†"€+‚€€€€€€€†"€,†"€†"€-†"€.‚ÿu Any SAT solution gives a solution to the IP problem - In any SAT solution, a literal corresponds to a 1 in the integer program, since the clause is satisfied. Therefore, the sum in each clause inequality is .u Any IP solution gives a SAT solution - In any solution to this integer programming instance, all variables must be set to either 0 or 1. If , then set literal . If , then set literal . No Boolean variable and its complement can both be true, so it is a legal assignment, which must also satisfy all the clauses.¹”8‡+šŠ+% €)€€‚ÿThe reduction works both ways, so integer programming must be hard. Notice the following properties, which hold true in general for NP-complete:>øá‰+؍+F Z€ñ€Ȁ€€€€‚€€‚€€€€‚ÿu The reduction preserved the structure of the problem. It did not solve the problem, just put it into a different format.u The possible IP instances that can result from this transformation are only a small subset of all possible IP instances. However, since some of them are hard, the general problem must be hard.u The transformation captures the essence of why IP is hard. It has nothing to do with having big coefficients or big ranges on variables; since restricting them to 0/1 is enough. It has nothing to do with having inequalties with large numbers of variables. Integer programming is hard because satisfying a set of constraints is hard. A careful study of the properties needed for a reduction can tell us a lot about the problem.,šŠ+Ž+' € €€€‚ÿ)؍+-Ž+& €€€‚ÿ¹žŽ+ À+ Ё€€‚ç‡f_ɀ†"€ ‰€çZf_ɀ†"€ ‰€çZf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÇÛ/€†"€‰€æÇÛ/€†"€‰€æÇÛ/€†"€‰€€ç‡f_ɀ‰€€çZf_ɀ‰€€çZf_ɀ‰€€‚‚ÿ Next: Vertex Cover Up: Difficult Reductions Previous: Difficult Reductions AlgorithmsMon Jun 2 23:33:50 EDT 1997-Ž+ À+¹G+)-Ž+5À+& €€€‚ÿµf À+êÀ+O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b15À+Á+1UÿÿÿÿÿÿÿÿéÿÿÿÿÁ+?Á+$êÀ+?Á+" €€€ ÿ= Á+|Á+1Ùÿÿÿÿÿÿÿÿê|Á+¯Á+Î,Vertex Cover3?Á+¯Á+# € €€‚ÿVertex Cover ƒf|Á+2Ä+ ‚߀€ç~f_ɀ†"€ ‰€çZf_ɀ†"€ ‰€ç[f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÇÛ/€†"€‰€æÇÛ/€†"€‰€æÇÛ/€†"€‰€€ç~f_ɀ‰€€çZf_ɀ‰€€ç[f_ɀ‰€‚€ ‚€ç?EÓš€†"€‰€€‚ÿ Next: Other NP-Complete Problems Up: Difficult Reductions Previous: Integer ProgrammingVertex CoverAlgorithmic graph theory proves to be a fertile ground for hard problems. The prototypical NP-complete graph problem is vertex cover, previously defined in Section as follows:   ^ì¯Á+Æ+r ²€Ý€€€€€€€€€†"€ï€‚€€€€€†"€ð€€€€€€‚ÿInput: A graph G=(V, E) and integer . Output: Is there a subset S of at most k vertices such that every has at least one vertex in S? Demonstrating the hardness of vertex cover proves more difficult than the previous reductions we have seen, because the structure of the two relevant problems is very different. A reduction from 3-satisfiability to vertex cover has to construct a graph G and bound k from the variables and clauses of the satisfiability instance.òƒ2Ä+‚É+o ¬€€€†"€†"€†"€/€€‚†"€0€€†"€1‚€€€€‚ÿFirst, we translate the variables of the 3-SAT problem. For each Boolean variable , we create two vertices and connected by an edge. To cover these edges, at least n vertices will be needed, since no two of the edges will share a vertex.  Figure: Reducing satisfiability instance to vertex cover  Second, we translate the clauses of the 3-SAT problem. For each of the c clauses, we create three new vertices, one for each literal in each clause. The three vertices of each clause will be connected so as to form c triangles. At least two vertices per triangle must be included in any vertex cover of these triangles.MäÆ+ÏÌ+i  €Í€€€€€€€€€€†"€1ç38€†"€‰€‚€€€€‚ÿFinally, we will connect these two sets of components together. Each literal in the vertex ``gadgets'' is connected to corresponding vertices in the clause gadgets (triangles) that share the same literal. From a 3-SAT instance with n variables and c clauses, this constructs a graph with 2n+3c vertices. The complete reduction for the 3-SAT problem is shown in Figure .This graph has been designed to have a vertex cover of size n+2c if and only if the original expression is satisfiable. By the earlier analysis, any vertex cover must have at least n+2c vertices, since adding extra edges to the graph can only increase the size of the vertex cover. To show that our reduction is correct, we must demonstrate that:c%‚É+2Ï+> J€M€Ȁ€€€€€†"€‚ÿu Every satisfying truth assignment gives a vertex cover - Given a satisfying truth assignment for the clauses, select the n vertices from the vertex gadgets that correspond to literals to be members of the vertex cover. Since this is a satisfying truth assignment, a true literal from each clause will have covered one of the three cross edges connecting each clause triangle to a vertex gadget. Therefore, by selecting the other two vertices of each clause triangle, we can also pick up the remaining cross edges and complete the cover.­dÏÌ+ë,I `€É€Ȁ€€€€€€€€€€€€‚ÿu Every vertex cover gives a satisfying truth assignment - Given any vertex cover C of size n+2c, exactly n of the vertices 2Ï+ë,?Á+must belong to the vertex gadgets. Let these first stage vertices define the truth assignment, while the 2c remaining cover vertices must be distributed at two per clause gadget; otherwise a clause gadget edge must go uncovered. These clause gadget vertices can cover only two of the three connecting cross edges per clause. Therefore, if C gives a vertex cover, at least one cross edge per clause must be covered, meaning that the corresponding truth assignment satisfies.Ù2Ï+,= H€µ€€ç?EÓš€†"€‰€€‚ÿThis proof of the hardness of vertex cover, chained with the clique and independent set arguments of Section , gives us a library of hard graph problems that we can use to make future hardness proofs easier.)ë,*,& €€€‚ÿÆÅ,ð, Ё›€€‚ç~f_ɀ†"€ ‰€çZf_ɀ†"€ ‰€ç[f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÇÛ/€†"€‰€æÇÛ/€†"€‰€æEÇÛ/€†"€‰€€ç~f_ɀ‰€€çZf_ɀ‰€€ç[f_ɀ‰€€‚‚ÿ Next: Other NP-Complete Problems Up: Difficult Reductions Previous: Integer Programming AlgorithmsMon Jun 2 23:33:50 EDT 1997)*,,& €€€‚ÿµfð,Î,O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1,ÿ,1Uÿÿÿÿÿÿÿÿëÿÿÿÿÿ,#,$Î,#," €€€ ÿKÿ,n,1ÿÿÿÿÿÿÿÿìn,¯,G,Other NP-Complete ProblemsA#,¯,# €<€€‚ÿOther NP-Complete Problems c^n, , ؁Í€€çf_ɀ†"€ ‰€çø7ۀ†"€ ‰€ç‡f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ<ÇÛ/€†"€‰€æ=ÇÛ/€†"€‰€æ>ÇÛ/€†"€‰€€çf_ɀ‰€€çø7ۀ‰€€ç‡f_ɀ‰€‚€‚€‚ÿ Next: The Art of Proving Up: Intractable Problems and Approximations Previous: Vertex CoverOther NP-Complete ProblemsClique, vertex cover, and integer programming are just three of the literally hundreds of problems that have been shown to be NP-complete. It is important to be aware of which kinds of problems tend to be hard, so you recognize them when you see them in applications, and also to provide a suitable class of candidates for future reductions. Some, but by no means all, of the hard problems from the catalog include:Cµ¯,U ,Ž ê€q€Ȁ€€€€€çú…¿J€†"€‰€‚€€€€€ç6 œú€†"€‰€‚€€€ç抏H€†"€‰€‚ÿu Integer partition - Can you partition n integers into two subsets such that the sums of the subsets are equal? See Section for details.u Bin packing - How many bins of a given size do you need to hold n items of variable size? See Section for details.u Chromatic number - How many colors do you need to color a graph such that no neighboring vertices are of the same color? See Section for details. œ ,a ,O l€€Ȁ€€€€†"€2çYS!€†"€‰€‚ÿu Bandwidth - Which permutation p of the vertices of a graph minimizes the length of the longest edge when the vertices are ordered on a line, i.e. ? See Section for details.QU ,²,O l€ €€çó‘¶ü€†"€‰€ç !‹€†"€‰€‚ÿA few other catalog problems exist in a limbo state, where it is not known whether the problem has a fast algorithm or is NP-complete. The most prominent of these are graph isomorphism (see Section ) and primality testing (see Section ). That this limbo list is so short is quite a tribute to the state of the art in algorithm design and the power of NP-completeness. For almost every important problem for which we do not know a fast algorithm, we have a good solid reason for why one doesn't exist.*a ,èB,' €€€‚‚ÿThe same should hold true for the pro²,èB,#,blems you encounter in your work. One way or another they should be resolved as being either hard or polynomial. Leaving them in a limbo state is a sure sign of a bush-league algorithm designer.It takes experience to be able to sense whether a problem is likely to be hard or not. Perhaps the quickest way to gain this experience is through careful study of the catalog. Note that slightly changing the wording of a problem can make the difference between it being polynomial or NP-complete. Finding the shortest path in a graph is easy, while finding the longest path in a graph is hard. Constructing a tour that visits all the edges once in a graph is easy, while constructing a tour that visits all the vertices once is hard.V²,>D,: B€9€€€€çՈK€‰€€‚ÿThe first thing to do when you suspect a problem might be NP-complete is look in Garey and Johnson's book Computers and Intractability [GJ79], which contains a list of several hundred problems known to be NP-complete. Likely you will find the problem you are interested in.)èB,gD,& €€€‚ÿÊÉ>D,1F, Ё£€€‚çf_ɀ†"€ ‰€çø7ۀ†"€ ‰€ç‡f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ?ÇÛ/€†"€‰€æ@ÇÛ/€†"€‰€æAÇÛ/€†"€‰€€çf_ɀ‰€€çø7ۀ‰€€ç‡f_ɀ‰€€‚‚ÿ Next: The Art of Proving Up: Intractable Problems and Approximations Previous: Vertex Cover AlgorithmsMon Jun 2 23:33:50 EDT 1997)gD,ZF,& €€€‚ÿµf1F,G,O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ZF,@G,1Uÿÿÿÿÿÿÿÿíÿÿÿÿ@G,dG,$G,dG," €€€ ÿL@G,°G,1Õÿÿÿÿÿÿÿÿî°G,òG,(Ã,The Art of Proving HardnessBdG,òG,# €>€€‚ÿThe Art of Proving Hardness ?:°G,1K, ؁…€€ç€f_ɀ†"€ ‰€çø7ۀ†"€ ‰€ç~f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æBÇÛ/€†"€‰€æCÇÛ/€†"€‰€æDÇÛ/€†"€‰€€ç€f_ɀ‰€€çø7ۀ‰€€ç~f_ɀ‰€‚€‚€‚ÿ Next: War Story: Hard Against Up: Intractable Problems and Approximations Previous: Other NP-Complete ProblemsThe Art of Proving HardnessProving that problems are hard is a skill. But once you get the hang of it, reductions can be surprisingly straightforward and pleasurable to do. Indeed, the dirty little secret of NP-completeness proofs is that they are usually easier to create than explain, in the same way that it is often easier to rewrite old code than it is to understand and modify it.[òG,°K,$ €¶€€‚ÿI offer the following advice to those seeking to prove the hardness of a given problem:yK1K,)L,. ,€–€Ȁ€€€‚ÿu Make your source problem as simple (i.e. restricted) as possible. -þ°K,VN,/ ,€ÿ€Ȁ†"€‚ÿNever try to use the general traveling salesman problem (TSP) as a source problem. Better, use Hamiltonian cycle, i.e. TSP where all the weights are 1 or . Even better, use Hamiltonian path instead of cycle, so you don't have to worry about closing up the path. Best of all, use Hamiltonian path on directed planar graphs where each vertex has total degree 3. All of these problems are equally hard, and the more you can restrict the problem that you are reducing, the less work your reduction has to do.]1)L,¿€,, &€c€Ȁ€€‚ÿAs another example, never try to use full satisfiability to prove hardness. Start with 3-satisfiability. In fact, you don't even have to use full 3-satisfiability. Instead, consider planar 3-satisfiability, where there is a way to draw the clauses as a graph in the plane such you can connect all instances of the same literal together without edges crossing. This property tendVN,¿€,dG,s to be useful in proving the hardness of geometric problems. All these problems are equally hard, and hence NP-completeness reductions using any of them are equally convincing._5VN,,* $€j€Ȁ€€‚ÿu Make your target problem as hard as possible. ŸŽ¿€,܂,0 .€€Ȁ‚€€€‚ÿDon't be afraid to add extra constraints or freedoms in order to make your problem more general. Perhaps your undirected graph problem can be generalized into a directed graph problem and can hence be easier to prove hard. Once you have a proof of hardness for the general problem, you can then go back and try to simplify the target.u Select the right source problem for the right reason. æ,è„,& €Í€Ȁ‚ÿSelecting the right source problem makes a big difference in how difficult it is to prove a problem hard. Although theoretically any particular problem is as good as another, this is the first and easiest place to go wrong. When faced with trying to prove that a problem is hard, some people fish around through dozens of problems, looking for the one that seems the best fit. These people are amateurs; odds are they never will recognize what they are looking for when they see it.%ÿ܂, †,& €ÿ€Ȁ‚ÿI use four and only four problems as candidates for my hard source problem. Limiting them to four means that I can know a lot about each of these problems, such as which variants of these problems are hard and which are soft. My favorite problems are:Ïè„,&ˆ,J b€Ÿ€‘€€€€€‚€€€‚€€€€€‚ÿu 3-SAT: The old reliable. When none of the three problems below seem appropriate, I go back to the original source.u Integer partition: This is the one and only choice for problems whose hardness seems to require using large numbers.u Vertex cover: This is the answer for any graph problems whose hardness depends upon selection. Chromatic number, clique, and independent set all involve trying to select the correct subset of vertices or edges.Ò †,*‰,2 2€¥€‘€€€€€€‚ÿu Hamiltonian path: This is my choice for any graph problem whose hardness depends upon ordering. If you are trying to route or schedule something, Hamiltonian path is likely your lever into the problem.qC&ˆ,›‰,. ,€†€Ȁ€€€‚ÿu Amplify the penalties for making the undesired selection. mG*‰,Œ,& €€Ȁ‚ÿMany people are too timid in their thinking about hardness proofs. You are trying to translate one problem into another, while keeping the problems as close to their original identities as as possible. The easiest way to do this is to be bold with your penalties, to punish anyone for trying to deviate from your intended solution. Your thinking should be, ``if you select this element, then you have to pick up this huge set that prevents you from finding an optimal solution.'' The sharper the consequences for doing what is undesired, the easier it is to prove if and only if.}S›‰,…Œ,* $€Š€Ȁ€€‚ÿu Think strategically at a high level, then build gadgets to enforce tactics. ÖŠŒ,[Ž,0 .€M€Ȁ‚€€€‚ÿYou should be asking yourself the following types of questions. ``How can I force that either A or B but not both are chosen?'' ``How can I force that A is taken before B?'' ``How can I clean up the things I did not select?'' After you have an idea of what you want your gadgets to do, you can worry about how to actually craft them.u When you get stuck, alternate between looking for an algorithm or a reduction. ±‹…Œ,À,& €€Ȁ‚ÿSometimes the reason you cannot prove hardness is that there exists an efficient algorithm to solve your problem! Techniques such as dynamic programming or reducing to polynomial-time graph problems such as matching or network flow sometimes yield surprising polynomial algorithms. Whenever you can't prove hardness, it likely pays to alter your opinion occasionally to keep yourse[Ž,À,dG,lf honest.,[Ž,DÀ,' € €€€‚ÿ)À,mÀ,& €€€‚ÿÝÜDÀ,JÂ, ЁÉ€€‚ç€f_ɀ†"€ ‰€çø7ۀ†"€ ‰€ç~f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ©ÎÛ/€†"€‰€æ ÎÛ/€†"€‰€æ¡ÎÛ/€†"€‰€€ç€f_ɀ‰€€çø7ۀ‰€€ç~f_ɀ‰€€‚‚ÿ Next: War Story: Hard Against Up: Intractable Problems and Approximations Previous: Other NP-Complete Problems AlgorithmsMon Jun 2 23:33:50 EDT 1997)mÀ,sÂ,& €€€‚ÿµfJÂ,(Ã,O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1sÂ,YÃ,1UÿÿÿÿÿÿÿÿïÿÿÿÿYÃ,}Ã,$(Ã,}Ã," €€€ ÿR!YÃ,ÏÃ,1§ÿÿÿÿÿÿÿÿðÏÃ,Ä,ÅB-War Story: Hard Against the ClockH%}Ã,Ä,# €J€€‚ÿWar Story: Hard Against the Clock ÉÄÏÃ,àÆ, ؁™€€çf_ɀ†"€ ‰€çø7ۀ†"€ ‰€çf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ¢ÎÛ/€†"€‰€æ£ÎÛ/€†"€‰€æ€ÎÛ/€†"€‰€€çf_ɀ‰€€çø7ۀ‰€€çf_ɀ‰€‚€‚€‚ÿ Next: Approximation Algorithms Up: Intractable Problems and Approximations Previous: The Art of ProvingWar Story: Hard Against the ClockMy class's attention span was running down like sand through an hourglass. Eyes were starting to glaze even in the front row. Breathing had become soft and regular in the middle of the room. Heads were tilted back and eyes shut in the rear.  -úÄ, É,3 4€õ€€‚çՈK€‰€‚ÿThere were fifteen minutes left to go in my lecture on NP-completeness, and I couldn't really blame them. They had already seen several reductions like the ones presented here, but NP-completeness reductions are easier to create than to understand or explain. They had to watch one being created in order to appreciate this.I reached for my trusty copy of Garey and Johnson's book [GJ79], which contains a list of over four hundred different known NP-complete problems in an appendix in the back.[2àÆ,hË,) €e€€‚‚‚ÿ``Enough of this!'' I announced loudly enough to startle those in the back row. ``NP-completeness proofs are routine enough that we can construct them on demand. I need a volunteer with a finger. Can anyone help me?''A few students in the front held up their hands. A few students in the back held up their fingers. I opted for one from the front row.``Select a problem at random from the back of this book. I can prove the hardness of any of these problems in the now twelve minutes remaining in this class. Stick your finger in and read me a problem.''†7 É,îÍ,O l€s€€‚€€‚€‚€€€†"€3†"€4‚ÿI had definitely gotten their attention. But I could have done that by offering to juggle chainsaws. Now I had to deliver results without cutting myself into ribbons.The student picked out a problem. ``OK, prove that Inequivalence of Programs with Assignments is hard,'' she said.  ``Huh? I've never heard of that problem before. What is it? Read me the entire description of the problem so I can write it on the board.'' The problem was as follows: Input: A finite set X of variables, two programs and , each a sequence of assignments of the form»hË,-a €{€€†"€5‚†"€6€€€€€‚€€€€€€€‚‚ÿwhere the are in X; and a value set V. Output: Is there an initial assignment of a value from V to each variable in X such that the two programs yield different final values for some variable in X?I looked at my watch. Ten minutes to go. But now everything was on the table. I was faced with a language problem. The input was two programs with variables, and I had to test to see whether they always do the samîÍ,-}Ã,e thing.rBîÍ,ˆ-0 .€‡€€‚†"€7‚ÿ``First things first. We need to select a source problem for our reduction. Do we start with integer partition? 3-satisfiability? Vertex cover or Hamiltonian path?''Since I had an audience, I tried thinking out loud. ``Our target is not a graph problem or a numerical problem, so let's start thinking about the old reliable: 3-satisfiability. There seem to be some similarities. 3-SAT has variables. This thing has variables. To be more like 3-SAT, we could try limiting the variables in this problem so they only take on two values, i.e. . Yes. That seems convenient.''‰Z--/ ,€µ€€‚‚€€‚ÿMy watch said nine minutes left. ``So, class, which way does the reduction go. 3-SAT to language or language to 3-SAT?''The front row correctly murmured, ``3-SAT to language.''``Right. So we have to translate our set of clauses into two programs. How can we do that? We can try to split the clauses into two sets and write separate programs for each of them. But how do we split them? I don't see any natural way how to do it, because eliminating any single clause from the problem might suddenly make an unsatisfiable formula satisfiable, thus completely changing the answer. Instead, let's try something else. We can translate all the clauses into one program, and then make the second program be trivial. For example, the second program might ignore the input and always outputs either only true or only false. This sounds better. Much better.''Òˆ-' -D V€«€€‚†"€8‚†"€†"€‚ÿI was still talking out loud to myself, which wasn't that unusual. But I had people listening to me, which was.``Now, how can we turn a set of clauses into a program? We want to know whether the set of clauses can be satisfied, or if there is an assignment of the variables such that it is true. Suppose we constructed a program to evaluate whether is satisfied. We can do it like this ....''It took me a few minutes worth of scratching before I found the right program. I assumed that I had access to constants for and , which seemed reasonable, in the sense that it shouldn't make the problem algorithmically harder. Once my proof worked, I could later worry about removing the extra assumption if desired.j-‘ -L h€B€€‚†"€9‚†"€:‚†"€;‚ÿ µ' -F -% €!€€‚ÿ``Great. Now I have a way to evaluate the truth of each clause. I can do the same thing to evaluate whether all the clauses are satisfied.''j‘ -° -L h€B€€‚†"€<‚†"€=‚†"€>‚ÿ CF -ó -0 .€)€€‚†"€?‚ÿNow the back of the classroom was getting excited. They were starting to see a ray of hope that they would get to leave on time. There were two minutes left in class.``Great. So now we have a program that can evaluate to be true if and only if there is a way to assign the variables so as to satisfy the set of clauses. We need a second program to finish the job. What about ? Yes, that is all we need. Our language problem asks whether the two programs always output the same thing, regardless of the possible variable assignments. If the clauses are satisfiable, that means that there must be an assignment of the variables such that the long program would output true. Testing whether the programs are equivalent is exactly the same as asking if the clauses are satisfiable.''é¿° -Ü-* "€€€‚€‚ÿI lifted my arms in triumph. ``And so, the problem is neat, sweet, and NP-complete.'' I got the last word out just before the bell rang.This exercise was so much fun that I repeated it the next time I taught the course, using a different randomly selected problem. The audio from my ultimately successful attempt to prove hardness is accessible from the CD-ROM, if you want to hear what the creation of an NP-completeness proof sounds like.)ó -@-& €€€Ü-@-}Ã,‚ÿÖÕÜ-çA- Ё»€€‚çf_ɀ†"€ ‰€çø7ۀ†"€ ‰€çf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ¥ÎÛ/€†"€‰€æŠÎÛ/€†"€‰€æ§ÎÛ/€†"€‰€€çf_ɀ‰€€çø7ۀ‰€€çf_ɀ‰€€‚‚ÿ Next: Approximation Algorithms Up: Intractable Problems and Approximations Previous: The Art of Proving AlgorithmsMon Jun 2 23:33:50 EDT 1997)@-B-& €€€‚ÿµfçA-ÅB-O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1B-öB-1UÿÿÿÿÿÿÿÿñÿÿÿÿöB-C-$ÅB-C-" €€€ ÿIöB-cC-1ÿÿÿÿÿÿÿÿòcC-¢C-ƒ-Approximation Algorithms?C-¢C-# €8€€‚ÿApproximation Algorithms ÌÇcC-nG- ؁Ÿ€€ç‚f_ɀ†"€ ‰€çø7ۀ†"€ ‰€ç€f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æšÎÛ/€†"€‰€æ&ÍÛ/€†"€‰€æÍÛ/€†"€‰€€ç‚f_ɀ‰€€çø7ۀ‰€€ç€f_ɀ‰€‚€‚€‚ÿ Next: Approximating Vertex Cover Up: Intractable Problems and Approximations Previous: War Story: Hard AgainstApproximation AlgorithmsFor the practical person, demonstrating that a problem is NP-complete is never the end of the line. Presumably, there was a reason why you wanted to solve it in the first place. That reason for wanting the solve it will not have gone away on being told that there is no polynomial-time algorithm. You still seek a program that solves the problem of interest. All you know is that you won't find one that quickly solves the problem to optimality in the worst case. You still have the following options:  {2¢C-éI-I `€e€Ȁ€€€‚€€€‚€€€€€‚ÿu Algorithms fast in the average case - Examples of such algorithms include backtracking algorithms with substantial pruning.u Heuristics - Heuristic methods like simulated annealing or greedy approaches can be used to find a solution with no requirement that it be the best one.u Approximation algorithms - The theory of NP-completeness does not stipulate that it is hard to get close to the answer, only that it is hard to get the optimal answer. With clever, problem-specific heuristics, we can often get provably close to the optimal answer.­†nG-–M-' € €€‚‚ÿApproximation algorithms return solutions with a guarantee attached, namely that the optimal solution can never be much better than this given solution. Thus you can never go too far wrong in using an approximation algorithm. No matter what your input instance is and how lucky you are, you are doomed to do all right. Further, approximation algorithms realizing provably good bounds often are conceptually simple, very fast, and easy to program.One thing that is usually not clear, however, is how well the solution from an approximation algorithm compares to what you might get from a heuristic that gives you no guarantees. The answer could be worse or it could be better. Leaving your money in a savings account in a bank guarantees you 3% interest without risk. Still, you likely will do much better putting your money in stocks than in the bank, even though performance is not guaranteed.  ‚ZéI-O-( €µ€€€‚ÿOne way to get the best of approximation algorithms and heuristics is to run both of them on the problem instance and pick the solution giving the better answer. This way, you get a solution that comes with a guarantee and a second chance to do even better. When it comes to heuristics for hard problems, sometimes you can have it both ways.)–M-AO-& €€€‚ÿ&O-gO-# €€€‚ÿŒLAO- €-@ P€˜€ȀçULú\€‰€‚€ç¶ÚŒ€‰€‚ÿu Approximating Vertex Cover u The Euclidean Traveling SalesmangO- €-C-&gO-2€-# €€€‚ÿ) €-[€-& €€€‚ÿÝÜ2€-8‚- ЁÉ€€‚ç‚f_ɀ†"€ ‰€çø7ۀ†"€ ‰€ç€f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÍÛ/€†"€‰€æÍÛ/€†"€‰€æ ÍÛ/€†"€‰€€ç‚f_ɀ‰€€çø7ۀ‰€€ç€f_ɀ‰€€‚‚ÿ Next: Approximating Vertex Cover Up: Intractable Problems and Approximations Previous: War Story: Hard Against AlgorithmsMon Jun 2 23:33:50 EDT 1997)[€-a‚-& €€€‚ÿµf8‚-ƒ-O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1a‚-Gƒ-1UÿÿÿÿÿÿÿÿóÿÿÿÿGƒ-kƒ-$ƒ-kƒ-" €€€ ÿKGƒ-¶ƒ-1ÿÿÿÿÿÿÿÿô¶ƒ-÷ƒ-îÆ-Approximating Vertex CoverAkƒ-÷ƒ-# €<€€‚ÿApproximating Vertex Cover œ•¶ƒ-“†- ܁;€€çƒf_ɀ†"€ ‰€çf_ɀ†"€ ‰€çf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ!ÍÛ/€†"€‰€æ"ÍÛ/€†"€‰€æ#ÍÛ/€†"€‰€€çƒf_ɀ‰€€çf_ɀ‰€€çf_ɀ‰€‚€ ‚€‚‚ÿ Next: The Euclidean Traveling Salesman Up: Approximation Algorithms Previous: Approximation AlgorithmsApproximating Vertex Cover As we have seen before, finding the minimum vertex cover of a graph is NP-complete. However, a very simple procedure can efficiently find a cover that is at most twice as large as the optimal cover:  Y×÷ƒ-ì‰-‚ Ҁ³€€‚€€€€€€‚†"€@‚†"€A‚€€€€‚€€€€€€‚€‚ÿ VertexCover(G=(V,E)) while do: Select an arbitrary edge Add both u and v to the vertex cover Delete all edges from E that are incident on either u or v. It should be apparent that this procedure always produces a vertex cover, since each edge is only deleted immediately after an incident vertex has been added to the cover. More interesting is the claim that any vertex cover must use at least half as many vertices as this one. Why? Consider just the edges selected by the algorithm. No two of these edges can share a vertex. Therefore, any cover of just these edges must include at least one vertex per edge, which makes it at least half the size of the greedy cover.lH“†-XŠ-$ €€€‚ÿThere are several interesting things to notice about this algorithm:»qì‰--J b€å€Ȁ€€€çOLú\€†"€‰€€€‚ÿu Although the procedure is simple, it is not stupid - Many seemingly smarter heuristics can give a far worse performance in the worst case. For example, why not modify the procedure above to select only one of the two vertices for the cover instead of both. After all, the selected edge will be equally well covered by only one vertex. However, consider the star-shaped graph of Figure . This heuristic will produce a two-vertex cover, while the single vertex heuristic can return a cover as large as n-1 vertices, should we get unlucky and repeatedly select the leaf instead of the center as the cover vertex.ZXŠ-¢-5 :€¶€Ȁ†"€B€€‚ÿ  Figure: Neglecting to pick the center vertex leads to a terrible vertex cover  …C-3Á-B R€‰€Ȁ€€€†"€C‚€€€‚ÿu Greedy isn't always the answer - Perhaps the most natural heuristic for this problem would repeatedly select and delete the vertex of highest remaining degree for the vertex cover. After all, this vertex will cover the largest number of possible edges. However, in the case of ties or near ties, this heuristic can go seriously astray and in the worst case can yield a cover that is times optimal.u Making a heuristic more complicated does not necessarily make it better - It is easy to complicate heuristics by adding more spe¢-3Á-kƒ-cial cases or details. For example, the procedure above does not specify which edge should be selected next. It might seem reasonable always to select the edge whose endpoints have highest degree. However, this does not improve the worst-case bound and just makes it more difficult to analyze.š}¢-ÛÂ-+ $€û€Ȁ€€‚ÿu A postprocessing cleanup step can't hurt - The flip side of designing simple heuristics is that they can often be modified to yield better-in-practice solutions without weakening the approximation bound. For example, a postprocessing step that deletes any unnecessary vertex from the cover can only improve things in practice, even though it won't help the worst-case bound.73Á-Ä-( €€€€‚ÿThe important property of approximation algorithms is relating the size of the solution produced directly to a lower bound on the optimal solution. Instead of thinking about how well we might do, we have to think about the worst case i.e. how badly we might perform.)ÛÂ-;Ä-& €€€‚ÿÕÔÄ-Æ- Ё¹€€‚çƒf_ɀ†"€ ‰€çf_ɀ†"€ ‰€çf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ$ÍÛ/€†"€‰€æ%ÍÛ/€†"€‰€æQÍÛ/€†"€‰€€çƒf_ɀ‰€€çf_ɀ‰€€çf_ɀ‰€€‚‚ÿ Next: The Euclidean Traveling Salesman Up: Approximation Algorithms Previous: Approximation Algorithms AlgorithmsMon Jun 2 23:33:50 EDT 1997);Ä-9Æ-& €€€‚ÿµfÆ-îÆ-O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b19Æ-Ç-1UÿÿÿÿÿÿÿÿõÿÿÿÿÇ-CÇ-$îÆ-CÇ-" €€€ ÿQ Ç-”Ç-1Óÿÿÿÿÿÿÿÿö”Ç-ÛÇ-€.The Euclidean Traveling SalesmanG$CÇ-ÛÇ-# €H€€‚ÿThe Euclidean Traveling Salesman ü”Ç-êÊ- ô €€ç„f_ɀ†"€ ‰€çf_ɀ†"€ ‰€ç‚f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æHÍÛ/€†"€‰€æIÍÛ/€†"€‰€æJÍÛ/€†"€‰€€ç„f_ɀ‰€€çf_ɀ‰€€ç‚f_ɀ‰€‚€ ‚€‚€€€€‚ÿ Next: Exercises Up: Approximation Algorithms Previous: Approximating Vertex CoverThe Euclidean Traveling Salesman   In most natural applications of the traveling salesman problem, direct routes are inherently shorter than indirect routes. For example, if the edge weights of the graph are ``as the crow flies'', straight-line distances between pairs of cities, the shortest path from x to y will always be to fly directly.      ÁZÛÇ-«Í-g œ€»€€†"€D€€‚†"€E€€€€€€ç³ÚŒ€†"€‰€‚ÿ  Figure: The triangle inequality typically holds in geometric and weighted graph problems.  The edge weights induced by Euclidean geometry satisfy the triangle inequality, which insists that for all triples of vertices u, v, and w. The reasonableness of this condition is shown in Figure . Note that the cost of airfares is an example of a distance function that violates the triangle inequality, since it is sometimes cheaper to fly through an intermediate city than to fly to the destination directly. TSP remains hard when the distances are Euclidean distances in the plane.[6êÊ-.% €m€€‚ÿWhenever a graph obeys the triangle inequality, we can approximate the optimal traveling salesman tour using minimum spanning trees. First, observe that the weight of a minimum spanning tree is a lower bound on the cost of the optimal tour. Why? Deleting any edge from a tour leaves a path, the total weight of which must be no greater than that of the original tour. This path has no cycles, and hence is a tree, which means its weight is at least that of the minimum spanning tree. Thus the minimum spanning tree cost gives a lower bound on the optimal tou«Í-.CÇ-r.  „D«Í-–.@ N€‹€€ç»$”€†"€‰€€€‚ÿConsider now what happens in performing a depth-first traversal of a spanning tree. Suppose we walk through each tree edge as we process it in a depth-first search. We will visit each edge twice, once going down the tree when exploring it and once going up after exploring the entire subtree. For example, in the depth-first search of Figure , we visit the vertices in order 1-2-1-3-5-8-5-9-5-3-6-3-1-4-7-10-7-11-7-4-1, thus using every tree edge exactly twice. Therefore, this tour has weight twice that of the minimum spanning tree, and hence at most twice optimal.  #Î.¹.U x€¡€€†"€F€€‚€€çÃß{+€†"€‰€€‚ÿ  Figure: A depth-first traversal of a spanning tree, with the shortcut tour  However, vertices will be repeated on this depth-first search tour. To remove the extra vertices, at each step we can take a shortest path to the next unvisited vertex. The shortcut tour for the tree above is 1-2-3-5-8-9-6-4-7-10-11-1. Because we have replaced a chain of edges by a single direct edge, the triangle inequality ensures that the tour can only get shorter. Thus the shortcut tour is within weight twice that of optimal. More complicated but better approximation algorithms for Euclidean TSP are mentioned in Section . No approximation algorithms exist for TSPs that do not satisfy the triangle inequality.  )–.â.& €€€‚ÿÀ¿¹.¢. Ё€€‚ç„f_ɀ†"€ ‰€çf_ɀ†"€ ‰€ç‚f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æKÍÛ/€†"€‰€æLÍÛ/€†"€‰€æMÍÛ/€†"€‰€€ç„f_ɀ‰€€çf_ɀ‰€€ç‚f_ɀ‰€€‚‚ÿ Next: Exercises Up: Approximation Algorithms Previous: Approximating Vertex Cover AlgorithmsMon Jun 2 23:33:50 EDT 1997)â.Ë.& €€€‚ÿµf¢.€.O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Ë.±.1Uÿÿÿÿÿÿÿÿ÷ÿÿÿÿ±.Õ.$€.Õ." €€€ ÿ: ±. .1¡ÿÿÿÿÿÿÿÿø .? .˜€.Exercises0 Õ.? .# €€€‚ÿExercises ÏÊ . . ؁¥€€çfn_ɀ†"€ ‰€çø7ۀ†"€ ‰€çƒf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æNÍÛ/€†"€‰€æOÍÛ/€†"€‰€æPÍÛ/€†"€‰€€çfn_ɀ‰€€çø7ۀ‰€€çƒf_ɀ‰€‚€‚€‚ÿ Next: Implementation Challenges Up: Intractable Problems and Approximations Previous: The Euclidean Traveling SalesmanExercises Yß? .g .z €¿€Ȁ€€€€€€€€€€‚€€€€€€€€€€€€€€€€€€‚ÿ1. Prove that the vertex cover problem (does there exist a subset S of k vertices in a graph G such that every edge in G is incident upon at least one vertex in S?) remains NP-complete even when all the vertices in the graph are restricted to have even degree.2. An instance of the set cover problem consists of a set X of n elements, a family F of subsets of X, and an integer k. The question is, do there exist k subsets from F whose union is X?¿s .&.L h€ì€Ȁ†"€G†"€H€€€€†"€I‚ÿFor example, if and , there does not exist a solution for k=2 but there does for k=3 (for example, ).Wg .}.? L€5€Ȁ‚€€†"€J†"€ì‚ÿProve that set cover is NP-complete with a reduction from vertex cover.3. The baseball card collector problem is as follows. Given packets , each of which contains a subset of that year's baseball cards, is it possible to collect all the year's cards by buying packets?g9&. @.. ,€t€Ȁ†"€K‚ÿFor example, if the players are and the packets are}. @.Õ.3}.?@.. ,€ €Ȁ†"€L‚ÿ€O @.¿@.1 2€ž€Ȁ€€€€‚ÿthere does not exist a solution for k=2 but there does for k=3, such as3?@.ò@.. ,€ €Ȁ†"€M‚ÿ6¿@.(C.3 4€€Ȁ‚€€€€‚ÿProve that the baseball card collector problem is NP-hard using a reduction from vertex cover.4. (*) An Eulerian cycle is a tour that visits every edge in a graph exactly once. An Eulerian subgraph is a subset of the edges and vertices of a graph that has an Eulerian cycle. Prove that the problem of finding the number of edges in the largest Eulerian subgraph of a graph is NP-hard. (Hint: the Hamiltonian circuit problem is NP-hard even if each vertex in the graph is incident upon exactly three edges.)ªgò@.ÒD.C T€Ï€Ȁ€€€€€€€€€€‚ÿ5. The low degree spanning tree problem is as follows. Given a graph G and an integer k, does G contain a spanning tree such that all vertices in the tree have degree at most k (obviously, only tree edges count towards the degree)? For example, in the following graph, there is no spanning tree such that all vertices have degree less than three.3(C.E.. ,€ €Ȁ†"€N‚ÿ8ìÒD.=G.L f€Ù€‘€€€€‚€€€€€€€€€€‚ÿ1. Prove that the low degree spanning tree problem is NP-hard with a reduction from Hamiltonian path.2. Now consider the high degree spanning tree problem, which is as follows. Given a graph G and an integer k, does G contain a spanning tree whose highest degree vertex is at least k? In the previous example, there exists a spanning tree of highest degree 8. Give an efficient algorithm to solve the high degree spanning tree problem, and an analysis of its time complexity.n1E.«H.= H€c€Ȁ€€€€€€€€‚ÿ6. (*) The problem of testing whether a graph G contains a Hamiltonian path is NP-hard, where a Hamiltonian path P is a path that visits each vertex exactly once. There does not have to be an edge in G from the ending vertex to the starting vertex of P, unlike in the Hamiltonian cycle problem. Ã=G.ËJ.] ˆ€‡€Ȁ€€€€€€€€€€‚€€€€€€€€‚ÿGiven a directed acyclic graph G (a DAG), give an O(n+m)-time algorithm to test whether or not it contains a Hamiltonian path. (Hint: think about topological sorting and DFS.)7. (**) The 2-SAT problem is, given a Boolean formula in 2-conjunctive normal form (CNF), decide whether the formula is satisfiable. 2-SAT is like 3-SAT, except that each clause can have only two literals. For example, the following formula is in 2-CNF:3«H.þJ.. ,€ €Ȁ†"€O‚ÿPËJ.L.? L€¡€Ȁ€€‚€€€€€€‚ÿGive a polynomial-time algorithm to solve 2-SAT.8. (*) It is an open question whether the decision problem ``Is integer n a composite number, in other words, not prime?" can be computed in time polynomial in the size of the input. Why doesn't the following algorithm suffice to prove it is in P, since it runs in O(n) time?ç…þJ.tM.b ’€€Ȁ‚€€‚†"€‚€€‚†"€P‚†"€‚ÿ PrimalityTesting(n) composite := for i := 2 to n-1 do if then composite := ,L. M.' € €€€‚ÿ)tM.ÉM.& €€€‚ÿåä M.®O. ЁÙ€€‚çfn_ɀ†"€ ‰€çø7ۀ†"€ ‰€çƒf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ|ÍÛ/€†"€‰€æsÍÛ/€†"€‰€ætÍÛ/€†"€‰€€çfn_ɀ‰€€çø7ۀ‰€€çƒf_ɀ‰€€‚‚ÿ Next: Implementation Challenges Up: Intractable Problems and Approximations Previous: The Euclidean Traveling Salesman AlgorithmsMon Jun 2 23:33:50 EDT 1997)ÉM.×O.& €€€‚ÿµf®O.˜€.O n€Ì€€Ì#EF("h×O.˜€.Õ.ttp://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1×O.ɀ.1Uÿÿÿÿÿÿÿÿùÿÿÿÿɀ.í€.$˜€.í€." €€€ ÿ@ɀ.-.1Iÿÿÿÿÿÿÿÿú-.c.{.Data Structures6í€.c.# €&€€‚ÿData Structures °«-.„. ؁g€€ç†f_ɀ†"€ ‰€çû7ۀ†"€ ‰€çgn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æuÍÛ/€†"€‰€ævÍÛ/€†"€‰€æwÍÛ/€†"€‰€€ç†f_ɀ‰€€çû7ۀ‰€€çgn_ɀ‰€‚€‚€‚ÿ Next: Dictionaries Up: A Catalog of Algorithmic Previous: CaveatsData StructuresData structures are not really algorithms that you can find and plug into your application.   Instead, they are the fundamental constructs for you to build your program around. Becoming fluent in what data structures can do for you is essential to get full value from them.<c.O†.% €/€€‚ÿBecause of this, this section is slightly out of sync with the rest of the catalog. Perhaps the most useful aspect of it will be the pointers to implementations of various data structures. Many of these data structures are nontrivial to implement well, so the programs we point to will likely be useful as models even if they do not do exactly what you need. Certain fundamental data structures, like kd-trees and suffix trees,   are not as well known as they should be. Hopefully, this catalog will serve to better publicize them.Šf„.ن.$ €Ì€€‚ÿThere is a large number of books on elementary data structures available. Our favorites include:  ³O†.òˆ.f š€i€Ȁ€€€çXŠK€‰€çöT9€†"€‰€‚€€€灞K€‰€‚ÿu Gonnet and Baeza-Yates [GBY91] - The book is a comprehensive reference to fundamental searching, sorting, and text searching algorithms. It features over 2,000 references and implementations in C and Pascal. These programs are now available by ftp/WWW. See Section for more details.u Weiss [Wei92] - A nice text, emphasizing data structures more than algorithms.       Comes in Pascal, C++, and Ada editions. Óن.ü‰.7 <€§€Ȁ€€籞K€‰€‚ÿu Wood [Woo93] - A thorough and accessible treatment of modern data structures, including suffix trees and geometric data structures. Pascal implementations for many of the data structures are provided.:òˆ.6‹.4 6€ €€ç‘K€‰€€‚ÿMehlhorn and Tsakalidis [MT90b] provide a detailed and up-to-date survey of research in data structures. The student who took only an elementary course in data structures is likely to be impressed by the volume and quality of recent work on the subject.)ü‰._‹.& €€€‚ÿ&6‹.…‹.# €€€‚ÿ˜_‹.žŒ. Ѐ1€Ȁçمـ‰€‚€ç…þ4.€‰€‚€çýùր‰€‚€ç}‹6€‰€‚€çÞ•€‰€‚€ç?š ô€‰€‚ÿu Dictionaries u Priority Queues u Suffix Trees and Arrays u Graph Data Structures u Set Data Structures u Kd-Trees&…‹.Č.# €€€‚ÿ)žŒ.íŒ.& €€€‚ÿ°¯ÄŒ.Ž. Ёo€€‚ç†f_ɀ†"€ ‰€çû7ۀ†"€ ‰€çgn_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æxÍÛ/€†"€‰€æyÍÛ/€†"€‰€æzÍÛ/€†"€‰€€ç†f_ɀ‰€€çû7ۀ‰€€çgn_ɀ‰€€‚‚ÿ Next: Dictionaries Up: A Catalog of Algorithmic Previous: Caveats AlgorithmsMon Jun 2 23:33:50 EDT 1997)íŒ.Ǝ.& €€€‚ÿµfŽ.{.O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Ǝ.¬.1Uÿÿÿÿÿÿÿÿûÿÿÿÿ¬. À.${. À." €€€ ÿ¬. À.{.= ¬.IÀ.1(@ÿÿÿÿÿÿÿÿüIÀ.|À.£/Dictionaries3 À.|À.# € €€‚ÿDictionaries úIÀ.—Â.! ‚€€ç²f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ{ÍÛ/€†"€‰€æ§ÍÛ/€†"€‰€æžÍÛ/€†"€‰€€ç²f_ɀ‰€€ç…f_ɀ‰€€ç…f_ɀ‰€‚€ ‚€‚†"€Q€‚€€€€‚ÿ Next: Priority Queues Up: Data Structures Previous: Data StructuresDictionaries    Input description: A set of n records, each identified by one or more key fields. î²|À.…Å.< F€e€€€€€€‚€€€‚ÿProblem description: Build and maintain a data structure to efficiently locate, insert, or delete the record associated with any query key q. Discussion: The abstract data type ``dictionary'' is one of the most important structures in computer science. Dozens of different data structures have been proposed for implementing dictionaries including hash tables, skip lists, and balanced/unbalanced binary search trees - so choosing the right one can be tricky. Depending on the application, it is also a decision that can significantly impact performance. In practice, it is more important to avoid using a bad data structure than to identify the single best option available.  ç—Â.lÈ.% €…€€‚ÿAn essential piece of advice is to carefully isolate the implementation of the dictionary data structure from its interface. Use explicit calls to subroutines that initialize, search, and modify the data structure, rather than embedding them within the code. This leads to a much cleaner program, but it also makes it easy to try different dictionary implementations to see how they impact performance. Do not obsess about the cost of the procedure call overhead inherent in such an abstraction. If your application is so time-critical that such overhead can impact performance, then it is even more essential that you be able to easily experiment with different implementations of your dictionary.    ‡c…Å.óÈ.$ €Æ€€‚ÿIn choosing the right data structure for your dictionary, ask yourself the following questions:%ìlÈ.Ë.9 @€Ù€Ȁ€€€‚€€€‚ÿu How many items will you typically have in your data structure? - Will you know this number in advance? Are you looking at a problem small enough that the simple data structure will be best, or will it be so large that we must worry about using too much memory or swapping?u Do you know the relative number of insertions, deletions, and search queries? - Will there be any modifications to the data structure after it is first constructed, or will it be static from that point on?#îóÈ.;Ï.5 8€Ý€Ȁ€€‚€€€‚ÿu Do you have an understanding of the relative frequency with which different keys will be accessed? - Can we assume that the access pattern will be uniform and random, or will it exhibit a skewed access distribution (i.e. certain elements are much more popular than others) or a sense of locality (i.e. elements are likely to be repeatedly accessed in clusters, instead of at fairly random intervals). Usually, the world is both skewed and clustered.      u Is it critical that individual operations be fast, or only that the total amount of work done over the entire program be minimized? - When response time is critical, such as in a program controlling a heart-lung machine, you can't wait too long between steps. When you have a program that is doing a lot of queries over the database, such as identifying all sex offenders who happen to be Republicans, it is not so critical that you pick out any particular congressman quickly as that you get them all with the minimum total effort.      iË.ÈÏ.$ €Ò€€‚ÿOnce you understand what your needs are, try to identify the best data structure from the list below:÷È;Ï.Ë// ,€‘€Ȁ€€€‚ÿu UnsÈÏ.Ë/ À.orted linked lists or arrays - For small data sets, say up to 10 to 20 items, an unsorted array is probably the easiest and most efficient data structure to maintain. They are easier to work with than linked lists, and if the dictionary will be kept this small, you cannot possibly save a significant amount of space over allocating a full array. If your dictionary will be too much larger, the search time will kill you in either case.        †ZÈÏ.Q/, &€µ€Ȁ€€‚ÿA particularly interesting and useful variant is a self-organizing list. Whenever a key is accessed or inserted, always move it to head of the list. Thus if the key is accessed again in the near future, it will be near the front and so require only a short search to find it. Since most applications exhibit both uneven access frequencies and locality of reference, the average search time for a successful search in a self-organizing list is typically much better than in a sorted or unsorted list. Of course, self-organizing data structures can be built from arrays as well as linked lists.    ÏŽË/ /A P€ €Ȁ€€‚€€€€€€€‚ÿu Sorted linked lists or arrays - Maintaining a sorted linked list is usually not worth the effort (unless you are trying to eliminate duplicates), since we cannot perform binary search in such a data structure. A sorted array will be appropriate if and only if there are not many insertions or deletions. When the array gets so large that it doesn't fit in real memory, think B-trees instead.        u Hash tables - For applications involving a moderate-to-large number of keys (say between 100 and 1,000,000), a hash table with bucketing is probably the right way to go. In a hash table, we use a function that maps keys (be they strings, numbers, or whatever) to integers between 0 and m-1. We maintain an array of m buckets, each typically implemented using an unsorted linked list. For a given key, the hash function immediately identifies which bucket will contain it. If we use a hash function that spreads the keys out nicely and a sufficiently large hash table, each bucket should contain very few items, thus making linear search acceptable. Insertion and deletion from a hash table reduce to insertion and deletion from the bucket/list.      É£Q/é /& €G€Ȁ‚ÿA well-tuned hash table will likely outperform a sorted array in most applications. However, several design decisions go into creating a well-tuned hash table:i' /R /B R€O€‘€€€€€€€€‚€€€‚ÿu How big should the table be? Typically, m should about the same as the maximum number of items you expect to put in the table. Make sure that m is a prime number, so as to minimize the dangers of a bad hash function.  u What hash function should I use? For strings, something like4é /† // .€ €‘€€†"€R‚ÿ׆R /] /Q p€€‘€€†"€S€€€€灒'ꀆ"€‰€‚ÿshould work, where is the size of the alphabet and char(x) is the function that maps each character x to its ASCII character code. For long strings, 8 to 10 characters should be sufficient to hash upon, provided they are unlikely to be padded blanks or some other invariant. Use Horner's rule to implement this hash function computation efficiently, as discussed in Section .*† /‡ /$ € €Ȁ‚ÿ   ò¶] /…A/< F€m€Ȁ€€‚€€€€€‚ÿRegardless of which hash function you decide to use, print statistics on the distribution of keys per bucket to see how uniform it really is. Odds are the first hash function you try will not prove to be the best. Botching up the hash function is an excellent way to slow down any application.u Binary search trees - Binary search trees are elegant data structures that support fast insertions, deletions, and queries. The big distinction between different types of trees is whether they are explicitly rebalanced after insertion or deletion, and how this rebalanci‡ /…A/ À.ng takes place. In random search trees, no rebalancing takes place and we simply insert a node at the leaf position where we can find it. Although search trees perform well under random insertions, most applications are not really random. Indeed, unbalanced search trees constructed by inserting keys in sorted order are a disaster, performing like a linked list.          _'‡ /äC/8 >€O€Ȁ€€€€€€‚ÿBalanced search trees use local rotation operations to restructure search trees, moving more distant nodes closer to the root while maintaining the in-order search structure of the tree. Among balanced search trees, AVL and 2/3 trees are now passé, and red-black trees seem to be more popular. A particularly interesting self-organizing data structure is the splay tree, which uses rotations to move any accessed key to the root. Frequently used or recently accessed nodes thus sit near the top of the tree, allowing fast search.            Ä”…A/šF/0 .€)€Ȁ‚€€€‚ÿBottom line: Which binary search tree is best for your application? Probably the balanced tree for which you have the best implementation readily available. See the choices below. Which flavor of balanced tree is probably not as important as how good the programmer was who coded it.u B-trees - For data sets so large that they will not fit in main memory (say more than 1,000,000 items) your best bet will be some flavor of a B-tree. As soon as the data structure gets outside of main memory, the search time to access a particular location on a disk or CD-ROM can kill you, since this is several orders of magnitude slower than accessing RAM.        $þäC/ÌH/& €ý€Ȁ‚ÿThe idea behind a B-tree is to collapse several levels of a binary search tree into a single large node, so that we can make the equivalent of several search steps before another disk access is needed. We can thereafter reference enormous numbers of keys using only a few disk accesses. To get the full benefit from using a B-tree, it is important to understand explicitly how the secondary storage device and virtual memory interact, through constants such as page size and virtual/real address space.    åšF/áJ/0 .€Ë€Ȁ‚€€€‚ÿEven for modest-sized data sets, unexpectedly poor performance of a data structure may be due to excessive swapping, so listen to your disk to help decide whether you should be using a B-tree.  u Skip lists - These are somewhat of a cult data structure. Their primary benefits seem to be ease of implementation relative to balanced trees. If you are using a canned tree implementation, and thus not coding it yourself, this benefit is eliminated. I wouldn't bother with them.  ‰ÌH/jN/l Š€?€€æŸÍÛ/€‰€€çƒ A€†"€‰€ç$ž;C€‰€‚çûJA€†"€‰€‚ÿImplementations: LEDA (see Section ) provides an extremely complete collection of dictionary data structures in C++, including hashing, perfect hashing, B-trees, red-black trees, random search trees, and skip lists. Given all of these choices, their default dictionary implementation is a randomized search tree [AS89], presumably reflecting which structure they expect to be most efficient in practice.      XTango (see Section ) is an algorithm animation system for UNIX and X-windows that includes animations of such dictionary data structures as AVL trees, binary search trees, hashing, red-black trees, and treaps (randomized search trees). Many of these are interesting and quite informative to watch. Further, the C source code for each animation is included.      ^áJ/Ԁ/H ^€/€€‚çXŠK€‰€çöT9€†"€‰€‚ÿThe 1996 DIMACS implementation challenge focused on elementary data structures like dictionaries. The world's best available implementations were likely to be identified during the course of the challenge, and they are accessible from http://dimacs.rutgers.edu/ .    Bare bones implementations in C and Pascal of a dizzying varietjN/Ԁ/ À.y of dictionary data structures appear in [GBY91], among them several variations on hashing and binary search trees, and optimal binary search tree construction. See Section for details.  Ú}jN/®„/] ˆ€û€€ç«ž;C€‰€€‚€ç‘K€‰€癑K€‰€çXŠK€‰€‚ÿImplementation-oriented treatments of a variety of dictionary data structures appear in [BR95], including hashing, splay trees, red-black trees, and what looks like a thorough implementation of B-trees. Code in C for these data structures is included in the text and is available on disk for a modest fee. Notes: Mehlhorn and Tsakalidis [MT90b] give a thorough survey of the state of the art in modern data structures. Knuth [Knu73a] provides a detailed analysis and exposition on fundamental dictionary data structures but misses such modern data structures as red-black and splay trees. Gonnet and Baeza-Yates [GBY91] provide implementations (in C and Pascal), detailed references, and experimental results for a wide variety of dictionary data structures. We defer to these sources to avoid giving original references for each of the data structures described above.ÌÙԀ/z‰/ó Ž³€€ç*‰K€‰€竞;C€‰€ç$ƒK€‰€籞K€‰€çJ˜K€‰€ç{˜K€‰€籞K€‰€箞;C€‰€竞;C€‰€ç$ƒK€‰€繐K€‰€籞K€‰€‚€€€€€€€€€€€€€€€€€€€€‚ÿGood expositions on red-black trees [GS78] include [BR95, CLR90, Woo93]. Good expositions on splay trees [ST85] include [Tar83, Woo93]. Good expositions on B-trees [BM72] include [BR95, CLR90]. Good expositions on hashing includes [Meh84, Woo93].Several modern data structures, such as splay trees, have been studied via amortized analysis, where we bound the total amount of time used by any sequence of operations. In an amortized analysis, we show that if a single operation is very expensive, this is because we have already benefited from enough cheap operations before it to pay off the higher cost. A data structure realizing an amortized complexity of O(f(n)) is less desirable than one whose worst-case complexity is O(f(n)) (since a very bad operation might still occur) but better than one with an average-case complexity O(f(n)), since the amortized bound will achieve this average on any input.    pþ®„/êŠ/r ²€€€ç$ž;C€‰€çK—K€‰€€‚€çÌAf€†"€‰€çfZHŀ†"€‰€€‚ÿNewer dictionary data structures that explicitly incorporate randomization into the construction include randomized search trees [AS89] and skip lists [Pug90].   Related Problems: Sorting (see page ), Searching (see page ).    )z‰/‹/& €€€‚ÿ²±êŠ/Ō/ Ёs€€‚ç²f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ ÍÛ/€†"€‰€æ¡ÍÛ/€†"€‰€æ¢ÍÛ/€†"€‰€€ç²f_ɀ‰€€ç…f_ɀ‰€€ç…f_ɀ‰€€‚‚ÿ Next: Priority Queues Up: Data Structures Previous: Data Structures AlgorithmsMon Jun 2 23:33:50 EDT 1997)‹/îŒ/& €€€‚ÿµfŌ/£/O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1îŒ/ԍ/1Uÿÿÿÿÿÿÿÿýÿÿÿÿԍ/ø/$£/ø/" €€€ ÿ@ԍ/8Ž/1"-ÿÿÿÿÿÿÿÿþ8Ž/nŽ/H0Priority Queues6ø/nŽ/# €&€€‚ÿPriority Queues !8Ž/›À/ ‚€€ç©f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç†f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ£ÍÛ/€†"€‰€æ€ÍÛ/€†"€‰€æ¥ÍÛ/€†"€‰€€ç©f_ɀ‰€€ç…f_ɀ‰€€ç†f_ɀ‰€‚€ ‚€‚†"€T€‚€€‚ÿ Next: Suffix Trees and Arrays Up: Data Structures PreviounŽ/›À/ø/s: DictionariesPriority Queues    Input description: A set of records with numerically or otherwise totally ordered keys. ¯snŽ/JÃ/< F€ç€€€€€€€€‚€‚ÿProblem description: Build and maintain a data structure for quickly inserting and deleting records, while enabling quick access to the smallest or largest key in the set. Discussion: Priority queues are useful data structures in simulations, particularly for maintaining a set of future events ordered by time so that we can quickly retrieve what the next thing to happen is. They are called ``priority'' queues because they enable you to retrieve items not by the insertion time (as in a stack or queue), nor by a key match (as in a dictionary), but by which item has the highest priority of retrieval.          -›À/wÅ/' € €€‚‚ÿIf your application performs no insertions after the first query, there is no need for an explicit priority queue. Simply sort the records by priority and proceed from top to bottom, maintaining a pointer to the last record deleted. This situation occurs in Kruskal's minimum spanning tree algorithm, or when simulating a completely scripted set of events.      However, if you are mixing insertions, deletions, and queries, you need a real priority queue. The following questions will help select the right one:åJÃ/•Ç/9 @€Ë€Ȁ€€€‚€€€‚ÿu Besides access to the smallest element, what other operations will you need? - Will you be searching for arbitrary keys, or just searching for the smallest? Will you be deleting arbitrary elements from the data, or just repeatedly deleting the top or smallest element?u Will you know the maximum size of your data structure in advance, or might an arbitrary number of items be inserted into it? - The issue here is whether you can preallocate space for the data structure.M"wÅ/âÈ/+ $€E€Ȁ€€‚ÿu Will you be changing the priority of elements already in the queue, or simply inserting and removing them? - Changing the priority of elements implies that we must be able to look up elements in the queue based on their key, in addition to being able to retrieve the largest element.xT•Ç/ZÉ/$ €š€€‚ÿDepending upon the answers, you have the following basic priority queue choices:N âÈ/šÌ/B R€€Ȁ€€€‚€€€†"€W‚ÿu Sorted array or list - In a sorted array,     it is very efficient to find and (by decrementing the top index) delete the smallest element. However, maintaining sortedness makes the insertion of new elements slow. Sorted arrays are suitable when there will be few insertions into the priority queue.u Binary heaps - This simple, elegant data structure supports both insertion and extract-min in time each.       Heaps maintain an implicit binary tree structure in an array, such that the key of the root of any subtree is less than that of all its descendents. Thus the minimum key is always at the root of the heap. New keys can be inserted by placing them at an open leaf and percolating the element upwards until it sits at its proper place in the partial order.·BZÉ/k0u ž€‡€Ȁ‚€€€€€€€€€€€€€€€€€€€†"€U€€€€‚ÿBinary heaps are the right answer whenever you know an upper bound on the number of items in your priority queue, since you must specify the array size at creation time.u Bounded height priority queue - This array-based data structure permits constant-time insertion and find-min operations whenever the range of possible key values is limited.   Suppose we know that all key values will be integers between 1 and n. We can set up an array of n linked lists, such that the ith list serves as a bucket containing all items with key i. We will maintain a pointer top to the smallest nonempty list. To insert an item with key k into the priority queue, add it to the kth bucket and set . To extract the minimum, reportšÌ/k0ø/ the first item from bucket top, delete it, and move top down if the bucket is now empty.GšÌ/²00 .€/€Ȁ‚€€€‚ÿBounded height priority queues are very useful in maintaining the vertices of a graph sorted by degree, which is a fundamental operation in graph algorithms.     Still, they are not as widely known as they should be. They are usually the right priority queue for any small, discrete range of keys.u Binary search trees - Binary search trees make effective priority queues, since the smallest element is always the leftmost leaf, while the largest element is always the rightmost leaf.     The min (max) is found by simply tracing down left (right) pointers until the next pointer is nil. Binary tree heaps prove most appropriate when you also need to search a dictionary of the values, or if you have an unbounded key range and do not know the maximum priority queue size in advance.͐k00= H€!€Ȁ€€€€€€€€‚ÿu Fibonacci and pairing heaps - These complicated priority queues are designed to speed up decrease-key operations, where the priority of an item already in the priority queue is reduced. This arises, for example, in shortest path computations whenever we discover a shorter route to a vertex v than we had previously established. Thus v has a higher priority of being accessed next.      úÔ²0y0& €©€Ȁ‚ÿProperly implemented and used, they lead to better performance on very large computations. Still, they are sufficiently complicated that you shouldn't mess with them unless you really know what you are doing.[ò0Ô 0i  €é€€æŠÍÛ/€‰€çƒ A€†"€‰€‚çҞ;C€‰€†"€瀂K€‰€‚ÿImplementations: LEDA (see Section ) provides a complete collection of priority queues in C++, including Fibonacci heaps, pairing heaps, Emde-Boas trees, and bounded height priority queues. Fibonacci heaps are their default implementation.  SimPack/Sim++ is a library of routines for implementing discrete event simulations, built by Robert Cubert and Paul Fishwick, of the University of Florida.     Priority queues are integral to such simulations, and Sim++ contains implementations of linked, binary, leftist, and calendar heaps [Bro88]. If you need a priority queue to control a simulation, check out http://www.cis.ufl.edu/ fishwick/simpack/simpack.html. An associated book [Fis95] describes model design using SimPack.    Söy0' 0] ˆ€ñ€€çXŠK€‰€çöT9€†"€‰€‚çûJA€†"€‰€‚ÿBare bones implementations in C and Pascal of the basic priority queue data structures appear in [GBY91]. Most notable is the inclusion of implementations of exotic priority queues such as P-trees and pagodas. See Section for further details.        XTango (see Section ) is an algorithm animation system for UNIX and X-windows, that includes animations of such advanced priority queue data structures as binomial and Fibonacci heaps, as well as a spiffy animation of heapsort.    Î Ô 0õ0à T€€ç‘K€‰€çam:7€†"€‰€çŠK€‰€çw' €†"€‰€€‚€ç(ž;C€‰€çOž;C€‰€ç$ƒK€‰€瑐K€‰€ç‘K€‰€çXŠK€‰€çŠK€‰€‚ÿMany textbooks provide implementations of simple priority queues, including [MS91] (see Section ). Algorithm 561 [Kah80] of the Collected Algorithms of the ACM is a Fortran implementation of a heap (see Section ).   Notes: Good expositions on efficient heap construction algorithms include [Baa88, Ben86, CLR90, Man89, MT90b]. See [GBY91] for a description of several exotic priority queues. Empirical comparisons between priority queue data structures include [Jon86].®' 0C0o ¬€a€€ç K€‰€†"€V€€€€‚çõ‚K€‰€€€†"€WçP˜K€‰€‚ÿBounded height priority queues are useful data structures in practice, but they do not have good worst-case performance bounds when arbitrary insertions aõ0C0ø/nd deletions are permitted. However, von Emde Boas priority queues [vEBKZ77] support insertion, deletion, search, max, and min operations where each key is an element from 1 to n.    Fibonacci heaps [FT87] support insert and decrease-key operations in O(1) amortized time, with amortized time extract-min and delete operations. The constant-time decrease-key operation leads to faster implementations of classical algorithms for shortest-paths, weighted bipartite-matching, and minimum-spanning-tree.   In practice, Fibonacci heaps are nontrivial to implement and have large constant factors associated with them. However, pairing heaps have been proposed to realize the same bounds with less overhead. Experiments with pairing heaps are reported in [SV87].   7•õ0UE0¢ 3€€çʂK€‰€€€çÿˆK€‰€†"€WçŸ;C€‰€€‚€çЅـ†"€‰€çÌAf€†"€‰€çUJN€†"€‰€€‚ÿHeaps define a partial order that can be built using a linear number of comparisons.   The familiar linear-time merging algorithm for heap construction is due to Floyd [Flo64]. In the worst case, 1.625n comparisons suffice [GM86] and comparisons are necessary [CC92]. Related Problems: Dictionaries (see page ), sorting (see page ), shortest path (see page ).      )C0~E0& €€€‚ÿ·¶UE05G0 Ё}€€‚ç©f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç†f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÒÍÛ/€†"€‰€æÉÍÛ/€†"€‰€æÊÍÛ/€†"€‰€€ç©f_ɀ‰€€ç…f_ɀ‰€€ç†f_ɀ‰€€‚‚ÿ Next: Suffix Trees and Arrays Up: Data Structures Previous: Dictionaries AlgorithmsMon Jun 2 23:33:50 EDT 1997)~E0^G0& €€€‚ÿµf5G0H0O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1^G0DH01UÿÿÿÿÿÿÿÿÿÿÿÿÿDH0hH0$H0hH0" €€€ ÿHDH0°H01Æ1ÿÿÿÿÿÿÿÿ°H0îH0&1Suffix Trees and Arrays>hH0îH0# €6€€‚ÿSuffix Trees and Arrays â°H0ñJ0! ‚×€€çªf_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç²f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æËÍÛ/€†"€‰€æÌÍÛ/€†"€‰€æÍÍÛ/€†"€‰€€çªf_ɀ‰€€ç…f_ɀ‰€€ç²f_ɀ‰€‚€ ‚€‚†"€X€‚€€€€‚ÿ Next: Graph Data Structures Up: Data Structures Previous: Priority QueuesSuffix Trees and Arrays    Input description: A reference string S. Œ5îH0­O0‡ ܀q€€€€€€€€‚€†"€çÍ8ˆ€†"€‰€€€€€€€€€ç;‚ý€†"€‰€‚ÿProblem description: Build a data structure for quickly finding all places where an arbitrary query string q is a substring of S. Discussion: Suffix trees and arrays are phenomenally useful data structures for solving string problems efficiently and with elegance. If you need to speed up a string processing algorithm from to linear time, proper use of suffix trees is quite likely the answer. Indeed, suffix trees are the hero of the war story reported in Section .       In its simplest instantiation, a suffix tree is simply a trie of the n strings that are suffixes of an n-character string S.     A trie is a tree structure, where each node represents one character, and the root represents the null string. Thus each path from the root represents a string, described by the characters labeling the nodes traversed. Any finite set of words defines a trie, and two words with common prefixes will branch off from each other at the first distinguishing character. Each leaf represents the end of a string. Figure illustrates a simple trie.Ë^ñJ0„‚0m š€¿€€†"€Y€€€€€€‚€€€€€€­O0„‚0hH0€€€€€€€€‚ÿ  Figure: A trie on strings the, their, there, was, and when  Tries are useful for testing whether a given query string q is in the set.     Starting with the first character of q, we traverse the trie along the branch defined by the next character of q. If this branch does not exist in the trie, then q cannot be one of the set of strings. Otherwise we find q in |q| character comparisons regardless of how many strings are in the trie. Tries are very simple to build (repeatedly insert new strings) and very fast to search, although they can be expensive in terms of memory.Ûf­O0_‡0u ž€Ñ€€€€€€€€€€€€€€‚†"€†"€€€€€€€€€‚ÿA suffix tree is simply a trie of all the proper suffixes of S. The suffix tree enables you to quickly test whether q is a substring of S, because any substring of S is the prefix of some suffix (got it?). The search time is again linear in the length of q.The catch is that constructing a full suffix tree in this manner can require time and, even worse, space, since the average length of the n suffices is n/2 and there is likely to be relatively little overlap representing shared prefixes. However, linear space suffices to represent a full suffix tree by being clever. Observe that most of the nodes in a trie-based suffix tree occur on simple paths between branch nodes in the tree. Each of these simple paths corresponds to a substring of the original string. By storing the original string in an array and collapsing each such path into a single node described by the starting and ending array indices representing the substring, we have all the information of the full suffix tree in only O(n) space. The output figure for this section displays a collapsed suffix tree in all its glory.ôµ„‚0S‰0? L€k€€‚ç.‰K€‰€瞁K€‰€‚ÿEven better, there exist linear-time algorithms to construct this collapsed tree that make clever use of pointers to minimize construction time. The additional pointers used to facilitate construction can also be used to speed up many applications of suffix trees.But what can you do with suffix trees? Consider the following applications. For more details see the books by Gusfield [Gus97] or Crochemore and Rytter [CR94]:’_‡0e‹0€ ΀+€Ȁ€€€†"€Z€€€€€€†"€Z†"€Z€€€€€€€€€€€€‚ÿu Find all occurrences of q as a substring of S - Just as with a trie, we can walk down from the root to find the node associated with q. The positions of all occurrences of q in S are represented by the descendents of , which can be identified using a depth-first search from .   For collapsed suffix trees, this takes O(|q|+k) time if there are k occurrences of q in S.ÇeS‰0,0b ’€Í€Ȁ€†"€[€€€‚€€€€€€€€€€€€€‚ÿu Longest substring common to a set T of strings -   Build a single collapsed suffix tree containing all suffixes of all strings, with each leaf labeled with its original string. In the course of doing a depth-first search on this tree, we can label each node with both the length of its common prefix and the number of distinct strings from T that are children of it. Thus the best node can be selected in linear time.u Find the longest palindrome in S -   A palindrome is a string that reads the same if the order of characters is reversed, such as madam. To find the longest palindrome in a string S, build a single suffix tree containing all suffixes of S and the reversal of S, with each leaf identified by its starting position. A palindrome is defined by any node in this tree that has forward and reversed children from the same position.Îee‹0Ä0i  €Ï€€‚€€€€€€€€†"€W†"€\€€€€€€€€‚ÿSince the linear time suffix tree construction algorithm is tricky, I recommend either starting from an e,0Ä0hH0xisting implementation or using a simple, potentially quadratic-time incremental-insertion algorithm to build a compressed suffix tree. Another good option is to use suffix arrays, discussed below.  Suffix arrays do most of what suffix trees do, while typically using four times less memory than suffix trees.   They are also easier to implement. A suffix array is basically just an array that contains all the n suffixes of S in sorted order. Thus a binary search of this array for string q suffices to locate the prefix of a suffix that matches q, permitting efficient substring search in string comparisons. In fact, only character comparisons need be performed on any query, since we can identify the next character that must be tested in the binary search.   For example, if the lower range of the search is cowabunga and the upper range is cowslip, all keys in between must share the same first three letters, so only the fourth character of any intermediate key must be tested against q.îº,0ôÆ04 6€w€€†"€€€‚ÿThe space savings of suffix arrays result because as with compressed suffix trees, it suffices to store pointers into the original string instead of explicitly copying the strings. Suffix arrays use less memory than suffix trees by eliminating the need for explicit pointers between suffixes since these are implicit in the binary search. In practice, suffix arrays are typically as fast or faster to search than suffix trees. Some care must be taken to construct suffix arrays efficiently, however, since there are characters in the strings being sorted. A common solution is to first build a suffix tree, then perform an in-order traversal of it to read the strings off in sorted order!  .Ä0"Ê0ž '€€æÎÍÛ/€‰€çÍ8ˆ€†"€‰€†"€‚çXŠK€‰€çöT9€†"€‰€€‚€çô‚K€‰€çXŠK€‰€ç՝;C€‰€‚ÿImplementations: Ting Chen's and Dimitris Margaritis's C language implementations of suffix trees, reported in the war story of Section , are available on the algorithm repository WWW site: http://www.cs.sunysb.edu/ algorith.  Bare bones implementations in C of digital and Patricia trie data structures and suffix arrays appear in [GBY91]. See Section for details.   Notes: Tries were first proposed by Fredkin [Fre62], the name coming from the central letters of the word ``retrieval''. A survey of basic trie data structures with extensive references appears in [GBY91].   Expositions on tries include [AHU83].)€ôÆ0KÌ0… ؀I€€ç€žK€‰€甐K€‰€磘K€‰€瞁K€‰€ç.‰K€‰€籞K€‰€çԝ;C€‰€çž;C€‰€‚ÿEfficient algorithms for suffix tree construction are due to Weiner [Wei73], McCreight [McC76], and Ukkonen [Ukk92]. Good expositions on these algorithms include Crochmore and Wytter [CR94] and Gusfield [Gus97]. Textbooks include [Woo93] and [AHU74], where they are called position trees.   Several applications of suffix trees to efficient string algorithms are discussed in [Apo85].TÖ"Ê0«1~ ʀ­€€çåK€‰€€€çXŠK€‰€‚€€€€€€箉K€‰€ç|˜K€‰€ç.‰K€‰€€‚ÿSuffix arrays were invented by Manber and Myers [MM90], although an equivalent idea called Pat trees due to Gonnet and Baeza-Yates appears in [GBY91].  The power of suffix trees can be further augmented by using a data structure for computing the least common ancestor of any pair of nodes x, y in a tree in constant time, after linear-time preprocessing of the tree.     The original data structure is due to Harel and Tarjan [HT84], but it was significantly simplified by Schieber and Vishkin [SV88]. Expositions include Gusfield [Gus97]. The least common ancestor (LCA) of two nodes in a suffix tree or trie defines the node representing the longest common prefix of the two associated strings.   Being able to answer such queries in constant time is amazing, and useful as a building bloKÌ0«1hH0ck for many other algorithms. The correctness of the LCA data structure is difficult to see; however, it is implementable and can perform well in practice. ŒhKÌ0g1T x€Ô€€€ç{ÜŠ€†"€‰€ç`€7€†"€‰€€‚ÿRelated Problems: string matching (see page ), longest common substring (see page ).    )«11& €€€‚ÿž·g1H1 Ё€€‚çªf_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç²f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÏÍÛ/€†"€‰€æÐÍÛ/€†"€‰€æÑÍÛ/€†"€‰€€çªf_ɀ‰€€ç…f_ɀ‰€€ç²f_ɀ‰€€‚‚ÿ Next: Graph Data Structures Up: Data Structures Previous: Priority Queues AlgorithmsMon Jun 2 23:33:50 EDT 1997)1q1& €€€‚ÿµfH1&1O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1q1W11UÿÿÿÿÿÿÿÿÿÿÿÿW1{1$&1{1" €€€ ÿFW1Á11Œ4ÿÿÿÿÿÿÿÿÁ1ý1’Å1Graph Data Structures<{1ý1# €2€€‚ÿGraph Data Structures _0Á1\1/ ,‚s€€ç«f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç©f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æýÍÛ/€†"€‰€æôÍÛ/€†"€‰€æõÍÛ/€†"€‰€€ç«f_ɀ‰€€ç…f_ɀ‰€€ç©f_ɀ‰€‚€ ‚€‚†"€]€‚€€€€‚€€€€‚ÿ Next: Set Data Structures Up: Data Structures Previous: Suffix Trees and ArraysGraph Data Structures    Input description: A graph G. Problem description: Give a flexible, efficient data structure to represent G. [ý1·1I `€'€€€€€€€çk€hl€†"€‰€‚ÿDiscussion: While there are several possible variations, the two basic data structures for graphs are adjacency matrices and adjacency lists. What these data structures actually are is discussed in Section .     The issues in deciding which one to use include:Ù\1¿ 1/ ,€³€Ȁ€€€‚ÿu How big will your graph be? -   How many vertices will it have, both typically and in the worse case? Ditto for the number of edges? If your graph has 100 vertices, your adjacency matrix contains 10,000 entries. If your graph has 1,000 vertices, your adjacency matrix contains 1,000,000 entries. If your graph has 10,000 vertices, your adjacency matrix contains 100,000,000 entries - so forget about it. Adjacency matrices work only for small or very dense graphs.žh·1w 1P n€Ó€Ȁ€€†"€‡‚€€€€€€€€€‚ÿu How dense will your graph be? - If the graph is very dense, meaning that a large fraction of the vertex pairs define edges, there is probably no compelling reason to use adjacency lists, since you will be doomed to using space, anyway.        u Which algorithms will you be implementing? - Certain algorithms are easier on adjacency matrices (such as all-pairs shortest path) and others on adjacency lists (such as most DFS-based algorithms). Adjacency matrices win for algorithms that repeatedly ask, ``Is (i,j) in G?'' However, most graph algorithms can be modified to eliminate such queries.   ,û¿ 1£11 0€÷€Ȁ€€€€‚ÿu Will you be modifying the graph over the course of your application, and if so, how? - Repeated edge insertions and (particularly) deletions argue for adjacency matrices, or perhaps for fancier versions of adjacency lists such as binary search trees. However, more likely than modifying the topology of graph is modifying the attributes of a vertex or edge of the graph, such as size, weight, or color.   Attributes are best handled as extra fields in the vertex or edge records of adjacency lists.uPw 1$B1% €¡€€‚ÿBuilding a good general-purpose graph type is surprisi£1$B1{1ngly tricky and difficult. For this reason, we suggest that you check out existing implementations (particularly LEDA) before hacking up your own. Note that it costs only time linear in the size of the larger data structure to convert between adjacency matrices and adjacency lists. This conversion is unlikely to be the bottleneck in any application, if you decide you want to use both data structures and have the space to store them. This usually isn't necessary but might prove simplest if you are confused about the alternatives.Rî£1vE1d –€á€€€€€€€€皧œ€†"€‰€ç|_Ã4€†"€‰€€‚ÿPlanar graphs are those that can be drawn in the plane so that no two edges cross.   Many graphs arising in applications are planar by definition, such as maps of countries, while others are planar by happenstance, like any tree.   Planar graphs are always sparse, since any n-vertex planar graph can have at most 3n-6 edges, so they should usually be represented by adjacency lists. If the planar drawing (or embedding) of the graph is fundamental to what is being computed, planar graphs are best represented geometrically. See Section for algorithms for constructing planar embeddings from graphs, and Section for algorithms maintaining graphs implicit in the arrangements of geometric objects like lines and polygons.      äº$B1ZG1* "€u€€€‚‚ÿHypergraphs are generalized graphs where each edge may link subsets of more than two vertices.   For example, suppose we want to represent who is on which Congressional committee.   The vertices of our hypergraph would be the individual congressmen, while each hyperedge would represent one committee. Such arbitrary collections of subsets of a set are naturally thought of as hypergraphs.Two basic data structures for hypergraphs are:?ÁvE1™K1~ ʀ…€Ȁ€€€†"€€€€€€€€€€€€€‚€€€€€€€€€€€‚ÿu Incidence matrices, which are analogous to adjacency matrices and require space, where m is the number of hyperedges.   Each row corresponds to a vertex, and each column to an edge, with a nonzero entry in M[i,j] iff vertex i is incident to edge j. For standard graphs, there are two nonzero entries in each column. The degree of each vertex governs the number of nonzero entries in each row.u Bipartite incidence structures, which are analogous to adjacency lists, and hence suited for sparse hypergraphs.   There is a vertex of the incidence structure associated with each edge and vertex of the hypergraphs, and an edge (i,j) in the incidence structure if vertex i of the hypergraph is in edge j of the hypergraph. Adjacency lists are typically used to represent this incidence structure. This bipartite incidence structure also provides a natural way to visualize the hypergraph, by drawing the associated bipartite graph.EZG1N1: B€€€çÕ•€†"€‰€‚ÿSpecial efforts must be taken to represent very large graphs efficiently. However, interesting problems have been solved on graphs with millions of edges and vertices.   The first step is to make your data structure as lean as possible, by packing your adjacency matrix as a bit vector (see Section ) or   removing extra pointers from your adjacency list representation. For example, in a static graph (no edge insertions or deletions) each edge list can be replaced by a packed array of vertex identifiers, thus eliminating pointers and saving potentially half the space.#é™K1G‚1: B€Õ€€ç…üˆé€†"€‰€‚ÿAt some point it may become necessary to switch to a hierarchical representation of the graph, where the vertices are clustered into subgraphs that are compressed into single vertices.   Two approaches exist for making such a hierarchical decomposition.   The first breaks things into components in a natural or application-specific way. For example, knowing that your graph is a map of roads and cities suggests a natural decomN1G‚1{1position - partition the map into districts, towns, counties, and states. The other approach runs a graph partition algorithm as in Section .   If you are performing the decomposition for space or paging reasons, a natural decomposition will likely do a better job than some naive heuristic for an NP-complete problem.   Further, if your graph is really unmanageably large, you cannot afford to do a very good job of algorithmically partitioning it. You should first verify that standard data structures fail on your problem before attempting such heroic measures.ó­N1:…1F Z€]€€æöÍÛ/€‰€çƒ A€†"€‰€‚ÿImplementations: LEDA (see Section ) provides the best implemented graph data type currently available in C++. If at all possible, you should use it. If not, you should at least study the methods it provides for graph manipulation, so as to see how the right level of abstract graph type makes implementing algorithms very clean and easy.   Although a general graph implementation like LEDA may be 2 to 5 times slower and considerably less space efficient than a stripped-down special-purpose implementation, you have to be a pretty good programmer to realize this performance improvement. Further, this speed is likely to come at the expense of simplicity and clarity.   `G‚1š‡1F Z€7€€ç2‰K€‰€盠uR€†"€‰€‚ÿGraphEd [Him94], written in C by Michael Himsolt, is a powerful graph editor that provides an interface for application modules and a wide variety of graph algorithms.   If your application demands interaction and visualization more than sophisticated algorithmics, GraphEd might be the right place to start, although it can be buggy. GraphEd can be obtained by anonymous ftp from forwiss.uni-passau.de (132.231.20.10) in directory /pub/local/graphed. See Section for more details on GraphEd and other graph drawing systems.ªn:…1DŠ1< F€ß€€ç9.4^€†"€‰€‚‚ÿThe Stanford Graphbase (see Section ) provides a simple but flexible graph data structure in CWEB, a literate version of the C language. It is instructive to see what Knuth does and does not place in his basic data structure, although we recommend LEDA as a better basis for further development.   LINK is an environment for combinatorial computing that provides special support for hypergraphs, including the visualization of hypergraphs.   Although written in C++, it provides a Scheme language interface for interacting with the graphs.   LINK is available from http://dimacs.rutgers.edu/Projects/LINK.html .Q€š‡1•1­ (M€€ç‘K€‰€çam:7€†"€‰€‚ç˜K€‰€皌:œ€†"€‰€€‚€笉K€‰€çx˜K€‰€ç$ƒK€‰€ç՝;C€‰€ç{˜K€‰€‚ÿAn elementary implementation of a ``lazy'' adjacency matrix in Pascal, which does not have to be initialized, appears in [MS91].     See Section .Simple graph data structures in Mathematica are provided by Combinatorica [Ski90], with a library of algorithms and display routines.     See Section . Notes: It was not until the linear-time algorithms of Hopcroft and Tarjan [HT73b, Tar72] that the advantages of adjacency list data structures for graphs became apparent. The basic adjacency list and matrix data structures are presented in essentially all books on algorithms or data structures, including [CLR90, AHU83, Tar83].  ÂkDŠ1cÀ1W |€×€€€€ç K€‰€‚€€çr‚K€‰€çíå¿é€‰€‚ÿAn interesting question concerns minimizing the number of bits needed to represent arbitrary graphs on n vertices, particularly if certain operations must be supported efficiently.   Such issues are discussed in [vL90b].Dynamic graph algorithms are essentially data structures that maintain quick access to an invariant (such as minimum spanning tree or connectivity) under edge insertion and deletion.     Sparsification [EGIN92] is a general and interesting approach to constructing dynamic graph algorithms. See [•1cÀ1{1ACI92] for an experimental study on the practicality of dynamic graph algorithms.jÒ•1ÍÂ1˜ þ€©€€ç:K€‰€ç9K€‰€珐K€‰€ç8K€‰€‚çSž;C€‰€€‚€çÕ•€†"€‰€ç…üˆé€†"€‰€€‚ÿHierarchically-defined graphs often arise in VLSI design problems, because designers make extensive use of cell libraries [Len90]. Algorithms specifically for hierarchically-defined graphs include planarity testing [Len89], connectivity [LW88], and minimum spanning trees [Len87].   The theory of hypergraphs is presented by Berge [Ber89].   Related Problems: Set data structures (see page ), graph partition (see page ).    )cÀ1öÂ1& €€€‚ÿŸœÍÂ1ŽÄ1 Ё‹€€‚ç«f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç©f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ÷ÍÛ/€†"€‰€æøÍÛ/€†"€‰€æùÍÛ/€†"€‰€€ç«f_ɀ‰€€ç…f_ɀ‰€€ç©f_ɀ‰€€‚‚ÿ Next: Set Data Structures Up: Data Structures Previous: Suffix Trees and Arrays AlgorithmsMon Jun 2 23:33:50 EDT 1997)öÂ1ÝÄ1& €€€‚ÿµfŽÄ1’Å1O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÝÄ1ÃÅ11UÿÿÿÿÿÿÿÿÿÿÿÿÃÅ1çÅ1$’Å1çÅ1" €€€ ÿDÃÅ1+Æ11 (ÿÿÿÿÿÿÿÿ+Æ1eÆ1;M2Set Data Structures:çÅ1eÆ1# €.€€‚ÿSet Data Structures 5ÿ+Æ1šÈ16 :‚€€ç¬f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€çªf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æúÍÛ/€†"€‰€æûÍÛ/€†"€‰€æüÍÛ/€†"€‰€€ç¬f_ɀ‰€€ç…f_ɀ‰€€çªf_ɀ‰€‚€ ‚€‚†"€^€‚€†"€_†"€`†"€a€‚ÿ Next: Kd-Trees Up: Data Structures Previous: Graph Data StructuresSet Data Structures    Input description: A universe of items and a collection of subsets , where . ‘:eÆ1+Ì1W |€{€€€†"€b†"€•†"€Ë€€€‚€€€‚ÿProblem description: Represent each subset so as to efficiently (1) test whether , (2) find the union or intersection of and , and (3) insert or delete members of S. Discussion: In mathematical terms, a set is an unordered collection of objects drawn from a fixed universal set. However, it is usually useful for implementation to represent each set in a single canonical order, typically sorted, so as to speed up or simplify various operations. Sorted order turns the problem of finding the union or intersection of two subsets into a linear-time operation - just sweep from left to right and see what you are missing.     It also makes possible element searching in sublinear time. Finally, printing the elements of a set in a canonical order paradoxically reminds us that order really doesn't matter.   yõšÈ1€Ï1„ րõ€€€€çЅـ†"€‰€†"€c†"€dçýùր†"€‰€ç"ÁD?€†"€‰€‚€€‚ÿWe distinguish sets from two other kinds of objects: strings and dictionaries. If there is no fixed-size universal set, a collection of objects is best thought of as a dictionary, as discussed in Section .   If the order does matter in a subset, i.e. if is not the same as , then your structure is more profitably thought of as a string, so see Sections and .    When each subset has cardinality exactly two, they form edges in a graph whose vertices are the universal set. A system of subsets with no restrictions on the cardinality of its members is called a hypergraph. It often can be profitable to consider whether your problem has a graph-theoretical analogy, like connected components or shortest path in a hypergraph.  tP+Ì1$2$ € €€‚ÿYour primary alternatives for representing arbitrary s€Ï1$2çÅ1ystems of subsets are:¶?€Ï1Ú2w Œ€ƒ€Ȁ€€€€€€€€€†"€e€€†"€f€€€€€€€€€€‚ÿu Bit vectors -   If your universal set U contains n items, an n-bit vector or array can represent any subset . Bit i will be 1 if , otherwise bit i is 0. Since only one bit is used per element, bit vectors can be very space efficient for surprisingly large values of |U|. Element insertion and deletion simply flips the appropriate bit. Intersection and union are done by ``and-ing'' or ``or-ing'' the bits together. The only real drawback of a bit vector is that for sparse subsets, it takes O(n) time to explicitly identify all members of S.  M"$2'2+ $€E€Ȁ€€‚ÿu Containers or dictionaries -     A subset can also be represented using a linked list, array, binary tree, or dictionary containing exactly the elements in the subset. No notion of a fixed universal set is needed for such a data structure. For sparse subsets, dictionaries can be more space and time efficient than bit vectors and easier to work with and program. For efficient union and intersection operations, it pays to keep the elements in each subset sorted, so a linear-time traversal through both subsets identifies all duplicates.‹IÚ2²2B R€•€€€€çkî:€†"€‰€‚‚ÿIn many applications, the subsets are all pairwise disjoint, meaning that each element is in exactly one subset.   For example, consider maintaining the connected components of a graph or the party affiliations of politicians.     Each vertex/hack is in exactly one component/party. Such a system of subsets is called a set partition.   Algorithms for constructing partitions of a given set are provided in Section .For data structures, the primary issue is maintaining a given set partition as things change over time, perhaps as edges are added or party members defect. The queries we are interested in include ``which set is a particular item in?'' and ``are two items in the same set?'' as we modify the set by (1) changing one item, (2) merging or unioning two sets, or (3) breaking a set apart.   Your primary options are:M'2ÿ 28 >€-€Ȁ€€€†"€W‚ÿu Dictionary with subset attribute -   If each item in a binary tree has associated a field recording the name of the subset it is in, set identification queries and single element modifications can be performed in the time it takes to search in the dictionary, typically . However, operations like performing the union of two subsets take time proportional to (at least) the sizes of the subsets, since each element must have its name changed. The need to perform such union operations quickly is the motivation for the ...8²27 27 <€€Ȁ€€€€€€‚ÿu Union-Find Data Structure - Suppose we represent a subset using a rooted tree, where each node points to its parent instead of its children.   Further, let the name of the subset be the name of the item that is the root. Finding out which subset we are in is simple, for we keep traversing up the parent pointers until we hit the root. Unioning two subsets is also easy. Just make the root of one of two trees point to the other, so now all elements have the same root and thus the same subset name.   qKÿ 2š2& €—€Ȁ‚ÿCertain details remain, such as which subset should be the ultimate root of a union, but these are described in most every algorithms text. Union-Find is a fast, extremely simple data structure that every programmer should know about. It does not support breaking up subsets created by unions, but usually this is not an issue.8ë7 2ìA2M h€Ù€€€‚æ(ÎÛ/€‰€çƒ A€†"€‰€‚‚ÿNeither of these options provides access to all of the items in a particular subset without traversing all the items in the set. However, both can be appropriately augmented with extra pointers if it is important that this operation be fast. Implementations:š2ìA2çÅ1 LEDA (see Section ) provides dictionary data structures to maintain sets and the union-find data structure to maintain set partitions, all in C++.    LINK is an environment for combinatorial computing that provides special support for hypergraphs, including visualization of hypergraphs.   Although written in C++, it provides an additional Scheme language interface for interacting with the graphs.   LINK is available from http://dimacs.rutgers.edu/Projects/LINK.html.QÖš2=D2{ Ā±€€ç‘K€‰€çam:7€†"€‰€çô»G¹€†"€‰€€‚€çs—K€‰€ç՝;C€‰€‚ÿMany textbooks contain implementations of the union-find data structure, including [MS91] (see Section ).   An implementation of union-find underlies any implementation of Kruskal's minimum spanning tree algorithm. Section contains a selection of minimum spanning tree codes.     Notes: Optimal algorithms for such set operations as intersection and union were presented in [Rei72]. Good expositions on set data structures include [AHU83].xáìA2µG2— ü€Ç€€ç_ŠK€‰€ç$ƒK€‰€ç‘K€‰€†"€g€€€€çy˜K€‰€çz˜K€‰€†"€hçԝ;C€‰€磗K€‰€‚ÿGalil and Italiano [GI91] survey data structures for disjoint set union.   Expositions on the union-find data structure appear in most algorithm texts, including [CLR90, MS91]. The upper bound of on m union-find operations on an n-element set is due to Tarjan [Tar75], as is a matching lower bound on a restricted model of computation [Tar79]. The inverse Ackerman function grows notoriously slowly, so this performance is close to linear. Expositions on the Ackerman bound include [AHU74]. An interesting connection between the worst-case of union-find and the length of Davenport-Schintzl sequences, a combinatorial structure that arises in computational geometry, is established in [SA95].     Î#=D2ƒJ2« $Q€€€€€€†"€i€€ç)ž;C€‰€€‚€ç `4"€†"€‰€çkî:€†"€‰€繿 耆"€‰€çô»G¹€†"€‰€€‚ÿThe power set of a set S is the collection of all subsets of S.   Explicit manipulation of power sets quickly becomes intractable due to their size.   Implicit representations of power sets in symbolic form becomes necessary for nontrivial computations. See [BCGR92] for algorithms for and computational experience with symbolic power set representations. Related Problems: Generating subsets (see page ), generating partitions (see page ), set cover (see page ), minimum spanning tree (see page ).        )µG2¬J2& €€€‚ÿ±°ƒJ2]L2 Ёq€€‚ç¬f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€çªf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÎÛ/€†"€‰€æ ÎÛ/€†"€‰€æ!ÎÛ/€†"€‰€€ç¬f_ɀ‰€€ç…f_ɀ‰€€çªf_ɀ‰€€‚‚ÿ Next: Kd-Trees Up: Data Structures Previous: Graph Data Structures AlgorithmsMon Jun 2 23:33:50 EDT 1997)¬J2†L2& €€€‚ÿµf]L2;M2O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1†L2lM21UÿÿÿÿÿÿÿÿÿÿÿÿlM2M2$;M2M2" €€€ ÿ9lM2ÉM21[,ÿÿÿÿÿÿÿÿÉM2øM2,3Kd-Trees/ M2øM2# €€€‚ÿKd-Trees çÉM2€2- (‚ကç­f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç«f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ"ÎÛ/€†"€‰€æ#ÎÛ/€†"€‰€æ$ÎÛ/€†"€‰€€ç­f_ɀ‰€€ç…f_ɀ‰€€ç«f_ɀ‰€‚€ ‚€‚†"€j€‚€€€€€€€€‚ÿ Next: Numerical Problems Up: Data Structures Previous: Set Data StructuresKd-Trees    Input description: A set S of n points in k dimøM2€2M2ensions. ‡WøM2Ÿ‚20 .€¯€€€€‚€‚ÿProblem description: Construct a tree that partitions the space by half-planes such that each point is contained in its own box-shaped region. Discussion: Although many different flavors of kd-trees have been devised, their purpose is always to hierarchically decompose space into a relatively small number of cells such that no cell contains too many input objects.     This provides a fast way to access any input object by position. We traverse down the hierarchy until we find the cell containing the object and then scan through the few objects in the cell to identify the right one.  }O€2…2. *€¡€€†"€š‚ÿTypical algorithms construct kd-trees by partitioning point sets. Each node in the tree is defined by a plane through one of the dimensions that partitions the set of points into left/right (or up/down) sets, each with half the points of the parent node.   These children are again partitioned into equal halves, using planes through a different dimension. Partitioning stops after levels, with each point in its own leaf cell. Alternate kd-tree construction algorithms insert points incrementally and divide the appropriate cell, although such trees can become seriously unbalanced.  ÕŸ‚2.‡2= H€«€€€€€€€€€€‚ÿThe cutting planes along any path from the root to another node defines a unique box-shaped region of space, and each subsequent plane cuts this box into two boxes. Each box-shaped region is defined by 2k planes, where k is the number of dimensions. Indeed, the `kd' in kd-tree is short for k-dimensional tree. In any search performed using a kd-tree, we maintain the current region defined by the intersection of these half-spaces as we move down the tree.i…2»‡2$ €Ò€€‚ÿDifferent flavors of kd-trees differ in exactly how the splitting plane is selected. Options include:y.‡24‹2^ Š€=€Ȁ€€€†"€k†"€l†"€k‚€€€‚€€€‚ÿu Cycling through the dimensions - partition first on , then before cycling back to .u Cut along the largest dimension - select the partition dimension so as to make the resulting boxes as square or cube-like as possible. Selecting a plane to partition the points in half does not mean selecting a splitter in the middle of the box-shaped regions, since all the points may be in the left side of the box.  u Quadtrees or Octtrees -     Instead of partitioning with single planes, use all axis-parallel planes that pass through a given partition point. In two dimensions, this means creating four child cells, in 3D this means eight child cells. Quadtrees seem particularly popular on image data, where leaf cells imply that all pixels in the regions have the same color.    Ó¬»‡22' €Y€€‚‚ÿNonorthogonal (i.e. not axis-parallel) cutting planes have also been used, although they make maintaining the cell boundaries more complicated.    Ideally, our partitions evenly split both the space (ensuring nice, fat, regular regions) and the set of points (ensuring a log height tree) evenly, but this can be impossible for a given point set. The advantages of fat cells become clear in many applications of kd-trees:  '@4‹2:Â2ç œ…€Ȁ€€€€€€€€€ç>|Ðò€†"€‰€‚€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€ç1u©>€†"€‰€‚ÿu Point location -   To identify which cell a query point q lies in, we start at the root and test which side of the partition plane contains q. By repeating this process on the appropriate child node, we travel the tree to find the leaf cell containing q in time proportional to its height. See Section for more on point location.u Nearest neighbor search -   To find the point in S closest to a query point q, we perform point location to find the cell c containing q. Since c is bordered by some 2:Â2M2point p, we can compute the distance d(p,q) from p to q. Point p is likely very close to q, but it might not be the single closest neighbor. Why? Suppose q lies right at the boundary of a cell. Then q's nearest neighbor might lie just to the left of the boundary in another cell. Thus we must traverse all cells that lie within a distance of d(p,q) of cell c and verify that none of them contain closer points. With nice, fat cells, very few cells should need to be tested. See Section for more on nearest neighbor search.2Ѝ2lÆ2b ’€£€Ȁ€€ç’°€†"€‰€‚€€€€€€€€€€€‚ÿu Range search -   Which points lie within a query box or region? Starting from the root, check to see whether the query region intersects or contains the cell defining the current node. If it does, check the children; if not, none of the leaf cells below this node can possibly be of interest. We quickly prune away the irrelevant portions of the space. See Section for more on range search.u Partial key search -     Suppose we want to find a point p in S, but we do not have full information about p. Say we are looking for someone of age 35 and height 5'8'' but of unknown weight in a 3d-tree with dimensions age, weight, and height. Starting from the root, we can identify the correct decendant for all but the weight dimension. To be sure we find the right point, we must search both children of this node. We are better off the more fields we know, but such partial key search can be substantially faster than checking all points against the key.b+:Â2ÎÈ27 <€W€€€€€€€€‚ÿKd-trees are most effective data structures for small and moderate numbers of dimensions, say from 2 up to maybe 20 dimensions. As the dimensionality increases, they lose effectiveness, primarily because the ratio of the volume of a unit sphere in k-dimensions shrinks exponentially compared to a unit cube in k-dimensions.    Thus exponentially many cells will have to be searched within a given radius of a query point, say for nearest-neighbor search. Also, the number of neighbors for any cell grows to 2k and eventually become unmanageable.¬=lÆ2zÍ2o ¬€}€€€‚æ%ÎÛ/€‰€€€€€€€€€ç‘K€‰€€€†"€m€€‚ÿThe bottom line is that you should try to avoid working in high-dimensional spaces, perhaps by discarding the least important dimensions. Implementations: Ranger is a tool for visualizing and experimenting with nearest neighbor and orthogonal range queries in high-dimensional data sets, using multidimensional search trees.   Four different search data structures are supported by Ranger: naive kd-trees, median kd-trees, nonorthogonal kd-trees, and the vantage point tree. For each of these, Ranger supports queries in up to 25 dimensions under any Minkowski metric.   It includes generators for a variety of point distributions in arbitrary dimensions.   Finally, Ranger provides a number of features to aid in visualizing multidimensional data, best illustrated by the accompanying video [MS93]. To identify the most appropriate projection at a glance, Ranger provides a matrix of all two-dimensional projections of the data set. Ranger is written in C,   runs on Silicon Graphics and HP workstations, and is available from the algorithm repository.   Æ^ÎÈ2L3h ž€¿€€‚çXŠK€‰€çöT9€†"€‰€€‚€瀗K€‰€çЗK€‰€‚ÿThe 1996 DIMACS implementation challenge focuses on data structures for higher-dimensional data sets. The world's best kd-tree implementations were likely to be identified in the course of the challenge, and they are accessible from http://dimacs.rutgers.edu/.    Bare bones implementations in C of kd-tree and quadtree data structures appear in [GBY91]. See Section for details on how to ftp them. Notes: The best reference on kd-trees and other spatial data structures are two volumes by Samet [Sam90a, Sam90b], inzÍ2L3M2 which all major variants are developed in substantial detail./ŠzÍ2{3‰ à€O€€çXž;C€‰€çЗK€‰€‚€€çI—K€‰€ç=‘K€‰€‚çú;C€‰€çû;C€€†"€€‰€€‚ÿBentley [Ben75] is generally credited with developing kd-trees, although they have the typically murky history associated with most folk data structures. The most reliable history is likely from Samet [Sam90b].An exposition on kd-trees for orthogonal range queries in two dimensions appears in [PS85].   Expositions of grid files and other spatial data structures include [NH93].  Algorithms that quickly produce a point provably close to the query point are a recent development in higher-dimensional nearest neighbor search.   A sparse weighted-graph structure is built from the data set, and the nearest neighbor is found by starting at a random point and walking greedily in the graph towards the query point. The closest point found during several random trials is declared the winner. Similar data structures hold promise for other problems in high-dimensional spaces. See [AM93, AMN 94]. ñ‡L3l3j ¢€€€€ç1u©>€†"€‰€ç>|Ðò€†"€‰€ç’°€†"€‰€€‚ÿRelated Problems: Nearest-neighbor search (see page ), point location (see page ), range search (see page ).      ){3•3& €€€‚ÿ¹žl3N3 Ё€€‚ç­f_ɀ†"€ ‰€ç…f_ɀ†"€ ‰€ç«f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ&ÎÛ/€†"€‰€æ'ÎÛ/€†"€‰€æSÎÛ/€†"€‰€€ç­f_ɀ‰€€ç…f_ɀ‰€€ç«f_ɀ‰€€‚‚ÿ Next: Numerical Problems Up: Data Structures Previous: Set Data Structures AlgorithmsMon Jun 2 23:33:50 EDT 1997)•3w3& €€€‚ÿµfN3,3O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1w3]31Uÿÿÿÿÿÿÿÿÿÿÿÿ]33$,33" €€€ ÿC]3Ä31*ÿÿÿÿÿÿÿÿÄ3ý3 €3Numerical Problems93ý3# €,€€‚ÿNumerical Problems èÑÄ3å 3 ü³€€ç®f_ɀ†"€ ‰€çû7ۀ†"€ ‰€ç¬f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æJÎÛ/€†"€‰€æKÎÛ/€†"€‰€æLÎÛ/€†"€‰€€ç®f_ɀ‰€€çû7ۀ‰€€ç¬f_ɀ‰€‚€‚€€€çʘK€‰€‚ÿ Next: Solving Linear Equations Up: A Catalog of Algorithmic Previous: Kd-TreesNumerical ProblemsIf most problems you encounter are numerical in nature, there is a good chance that you are reading the wrong book. Numerical Recipes [PFTV86] gives a terrific overview to the fundamental problems in numerical computing, including linear algebra, numerical   integration, statistics, and differential equations. Different flavors of the book include source code for all the algorithms in C, Pascal, and Fortran. Their coverage is skimpier on the combinatorial/numerical problems we consider in this section, but you should be aware of that book.švý3 3$ €ì€€‚ÿNumerical algorithms tend to be different beasts than combinatorial algorithms, for at least two distinct reasons:ä©å 3c3; D€S€Ȁ€€€ç&˜K€‰€‚ÿu Issues of Precision and Error - Numerical algorithms typically perform repeated floating-point computations, which accumulate error at each operation until, eventually, the results are meaningless. An amusing example [SK93] concerns the Vancouver Stock Exchange, which over a twenty-two month period accumulated sufficient round-off error to reduce its index from the correct value of 1098.982 to 574.081.       íœ 3\B30 .€{€Ȁ‚€€€‚ÿA simple and dependable way to test for round-off errors in numerical     programs is to run them both at sc3\B33ingle and double precision, and then think hard whenever there is a disagreement.u Extensive Libraries of Codes -       Large, high-quality libraries of numerical routines have existed since the 1960s, which is still not the case for combinatorial algorithms. This is true for several reasons, including (1) the early emergence of Fortran as a standard for numerical computation, (2) the nature of numerical computations to remain independent rather than be embedded within large applications, and (3)   the existence of large scientific communities needing general numerical libraries.@c3œC3; D€ €Ȁçw' €†"€‰€‚ÿRegardless of why, you should exploit this software base. There is probably no reason to implement algorithms for any of the problems in this section instead of stealing existing codes.   Searching Netlib (see Section ) is always a good place to start.a4\B3ýE3- (€i€€‚€€‚ÿMost scientist's and engineer's ideas about algorithms derive from Fortran programming and numerical methods, while computer scientists grew up programming with pointers and recursion, and so are comfortable with the more sophisticated data structures required for combinatorial algorithms. Both sides can and should learn from each other, since several problems such as pattern recognition can be modeled either numerically or combinatorially.There is a vast literature on numerical algorithms. In addition to Numerical Recipes, recommended books include:(ÁœC3%I3g œ€ƒ€Ȁ€€€ç&˜K€‰€‚€€€çL—K€‰€‚€€€ç!ƒK€‰€‚ÿu Skeel and Keiper [SK93] - A readable and interesting treatment of basic numerical methods, avoiding overly detailed algorithm descriptions through its use of the computer algebra system Mathematica.   I like it.u Pizer and Wallace [PW83] - A numerical analysis book written for computer scientists, not engineers. The organization is by issues instead of problems. A different but interesting perspective.u Cheney and Kincaid [CK80] - A traditional Fortran-based numerical analysis text, with discussions of optimization and Monte Carlo methods in addition to such standard topics as root-finding, numerical integration, linear systems, splines, and differential equations.   í¶ýE3J37 <€m€Ȁ€€çûž;C€‰€‚ÿu Buchanan and Turner [BT92] - Thorough language-independent treatment of all standard topics, including parallel algorithms. Most comprehensive of the texts described here.,%I3>J3' € €€€‚ÿ)J3gJ3& €€€‚ÿ&>J3J3# €€€‚ÿWgJ3¥L3Á P¯€ȀçŲ€‰€‚€çbS!€‰€‚€çÃá'p€‰€‚€ç$p.π‰€‚€çÐèN$€‰€‚€çgç̀‰€‚€çÈu,€‰€‚€ç)!‹€‰€‚€犒'ꀉ€‚€çë .I€‰€‚ÿu Solving Linear Equations u Bandwidth Reduction u Matrix Multiplication u Determinants and Permanents u Constrained and Unconstrained Optimization u Linear Programming u Random Number Generation u Factoring and Primality Testing u Arbitrary-Precision Arithmetic u Knapsack Problem S#J3øL30 0€F€ȀçL¯4š€‰€‚ÿu Discrete Fourier Transform&¥L3M3# €€€‚ÿ)øL3GM3& €€€‚ÿœŒM3O3 Ё‰€€‚ç®f_ɀ†"€ ‰€çû7ۀ†"€ ‰€ç¬f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æMÎÛ/€†"€‰€æNÎÛ/€†"€‰€æOÎÛ/€†"€‰€€ç®f_ɀ‰€€çû7ۀ‰€€ç¬f_ɀ‰€€‚‚ÿ Next: Solving Linear Equations Up: A Catalog of Algorithmic Previous: Kd-Trees AlgorithmsMon Jun 2 23:33:50 EDT 1997)GM3-O3& €€€‚ÿµfO3 €3O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b-O3 €331-O3=€31Vÿÿÿÿÿÿÿÿ ÿÿÿÿ=€3a€3$ €3a€3" €€€ ÿI=€3ª€31º)ÿÿÿÿÿÿÿÿ ª€3é€3€4Solving Linear Equations?a€3é€3# €8€€‚ÿSolving Linear Equations w2ª€3`ƒ3E X‚{€€ç¯f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æPÎÛ/€†"€‰€æQÎÛ/€†"€‰€æRÎÛ/€†"€‰€€ç¯f_ɀ‰€€ç­f_ɀ‰€€ç­f_ɀ‰€‚€ ‚€‚†"€n€‚€†"€o€€†"€p€€€€€€€‚ÿ Next: Bandwidth Reduction Up: Numerical Problems Previous: Numerical ProblemsSolving Linear Equations    Input description: An matrix A, and an vector b, together representing m linear equations on n variables. „9é€3ä…3K d€u€€€€€†"€q€‚€çʁK€‰€‚ÿProblem description: What is the vector x such that ? Discussion: The need to solve linear systems arises in an estimated 75% of all scientific computing problems [DB74].   For example, applying Kirchhoff's laws to analyze electric circuits generates a system of equations, the solution of which gives currents through each branch of the circuit. Analysis of the forces acting on a mechanical truss generates a similar set of equations. Even finding the point of intersection between two or more lines reduces to solving a (small) linear system.       /Ì`ƒ3‰3c ”€™€€€€€€€€€€€€€€€€€€€€€€‚‚ÿNot all systems of equations have solutions; consider the equations 2x+3y = 5 and 2x+3y = 6. Some systems of equations have multiple solutions; consider the equations 2x+3y=5 and 4x+6y=10.   Such degenerate systems of equations are called singular, and they can be recognized by testing whether the determinant of the coefficient matrix is zero.    Solving linear systems is a problem of such scientific and commercial importance that excellent codes are readily available. There is likely no good reason to implement your own solver, even though the basic algorithm (Gaussian elimination) is one we learned in high school.     This is especially true if you are working with large systems.ä…3£‹3y À€/€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚ÿGaussian elimination is based on the fact that the solution to a system of linear equations is invariant under scaling (multiplying both sides by a constant; i.e. if x=y, then 2x=2y) and adding equations (i.e. the solution to the equations x=y and w=z is the same as the solution to x=y and x+w=y+z).   Gaussian elimination scales and adds equations so as to eliminate each variable from all but one equation, leaving the system in such a state that the solution can just be read off from the equations. »‰3¯3Q p€{€€†"€K†"€Q€€€€€€€€‚‚ÿThe time complexity of Gaussian elimination on an system of equations is , since for the ith variable we add a scaled copy of the n-term ith row to each of the n-1 other equations. On this problem, however, constants matter. Algorithms that only partially reduce the coefficient matrix and then backsubstitute to get the answer use 50% fewer floating-point operations than the naive algorithm.  Issues to worry about include:q<£‹3 35 8€y€Ȁ€€€€€‚ÿu Are roundoff errors and numerical stability affecting my solution? -     Implementing Gaussian elimination would be quite straightforward except for round-off errors, which accumulate with each row operation and can quickly wreak havoc on the solution, particularly with matrices that are almost singular.S¯3«À3, &€§€Ȁ€€‚ÿTo eliminate the danger of numerical errors, it pays to substitute the solution back into each of the original equations and test how close they are to the desired value.   Iter 3«À3a€3ative methods for solving linear systems refine initial solutions to obtain more accurate answers - good linear systems packages will include such routines.T$ 3ÿÃ30 .€I€Ȁ‚€€€‚ÿThe key to minimizing roundoff errors in Gaussian elimination is selecting the right equations and variables to pivot on, and to scale the equations so as to eliminate large coefficients.     This is an art as much as a science, which is why you should use one of the many well-crafted library routines described below.u Which routine in the library should I use? - Selecting the right code is also somewhat of an art. If you are taking your advice from this book, you should start with the general linear system solvers. Hopefully they will suffice for your needs. But search through the manual for more efficient procedures for solving special types of linear systems. If your matrix happens to be one of these special types, the solution time can reduce from cubic to quadratic or even linear.Áo«À3ÀÆ3R r€á€Ȁ€€€€€€€€çYS!€†"€‰€‚ÿu Is my system sparse? -   The key to recognizing that you might have a special-case linear system is establishing how many matrix elements you really need to describe A. If there are only a few nonzero elements, your matrix is sparse and you are in luck. If these few nonzero elements are clustered near the diagonal, your matrix is banded and you are in even more luck.   Algorithms for reducing the bandwidth of a matrix are discussed in Section .   Many other regular patterns of sparse matrices can also be exploited, so see the manual of your solver or a better book on numerical analysis for details.0ÆÿÃ3ðÈ3j ¢€“€Ȁ€€†"€q€€€€€€€€€€€€†"€r†"€q‚ÿu Will I be solving many systems with the same coefficient matrix? - In certain applications, such as least-squares curve fitting and differential equations, we have to solve repeatedly with different b vectors. For efficiency, we seek to preprocess A to make this easier.       The lower-upper or LU-decomposition of A creates lower- and upper-triangular matrices L and U such that . We can use this decomposition to solve , since3ÀÆ3#É3. ,€ €Ȁ†"€s‚ÿIððÈ3lÊ3Y €€ë€Ȁ†"€t†"€u€€†"€†"€Q†"€Q‚ÿThis is efficient since backsubstitution solves a triangular system of equations in quadratic time. Solving and then gives the solution x using two steps instead of one step, once the LU-decomposition has been found in time.ÁC#É3-Í3~ ʀ€€†"€v†"€wçp.π†"€‰€€‚æ~ÎÛ/€‰€ç‚K€‰€çw' €†"€‰€‚ÿThe problem of solving linear systems is equivalent to that of matrix inversion, since , where is the identity matrix.     However, avoid it, since matrix inversion proves to be three times slower than Gaussian elimination. LU-decompositions prove useful in inverting matrices as well as computing determinants (see Section ). Implementations: The library of choice for solving linear systems is apparently LAPACK, a descendant of LINPACK [DMBS79]. Both of these Fortran codes, as well as many others, are available from Netlib. See Section .        <ÏlÊ3iÏ3m š€¡€€ç÷—K€‰€ç՞;C€‰€ç‚K€‰€ç@‘K€‰€çw' €†"€‰€€‚ÿAlgorithm 533 [She78], Algorithm 576 [BS81], and Algorithm 578 [DNRT81] of the Collected Algorithms of the ACM are Fortran codes for Gaussian elimination.   Algorithm 533 is designed for sparse systems, algorithm 576 to minimize roundoff errors, and algorithm 578 to optimize virtual memory performance.   Algorithm 645 [NW86] is a Fortran code for testing matrix inversion programs. See Section for details on fetching these programs. zî-Í3ï4Œ æ€á€€€çʘK€‰€‚†"€Qçƒ A€†"€‰€€‚€ç$ƒK€‰€çûž;C€‰€ç!ƒK€‰€ç&˜K€‰€‚ÿNumericaliÏ3ï4a€3 Recipes [PFTV86] provides routines for solving linear systems.   However, there is no compelling reason to use these ahead of the free codes described in this section.C++ implementations of algorithms to solve linear equations and invert matrices are embedded in LEDA (see Section ).     Notes: Good expositions on algorithms for Gaussian elimination and LU-decomposition include [CLR90] and a host of numerical analysis texts [BT92, CK80, SK93].3ÂiÏ3"4q °€‡€€çf‘K€‰€‚çԝ;C€‰€ç$ƒK€‰€‚€€€€€€€€†"€x€‚ÿParallel algorithms for linear systems are discussed in [Ort88]. Solving linear systems is one of relatively few problems where parallel architectures are widely used in practice.  Matrix inversion, and hence linear systems solving, can be done in matrix multiplication time using Strassen's algorithm plus a reduction.   Good expositions on the equivalence of these problems include [AHU74, CLR90].Certain types of nonsparse systems can be solved efficiently via special algorithms.   In particular, Toeplitz matrices are constructed so that all the elements along a particular diagonal are identical, and Vandermonde matrices   are defined by an n-element vector x where . ¿kï4á4T x€Ú€€€çºá'p€†"€‰€çp.π†"€‰€€‚ÿRelated Problems: Matrix multiplication (see page ), determinant/permanent (see page ).    )"4 4& €€€‚ÿŒ»á4Æ4 Ё‡€€‚ç¯f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æuÎÛ/€†"€‰€ævÎÛ/€†"€‰€æwÎÛ/€†"€‰€€ç¯f_ɀ‰€€ç­f_ɀ‰€€ç­f_ɀ‰€€‚‚ÿ Next: Bandwidth Reduction Up: Numerical Problems Previous: Numerical Problems AlgorithmsMon Jun 2 23:33:50 EDT 1997) 4ï4& €€€‚ÿµfÆ4€4O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ï4Õ41Uÿÿÿÿÿÿÿÿ ÿÿÿÿÕ4ù4$€4ù4" €€€ ÿDÕ4= 41¹ ÿÿÿÿÿÿÿÿ = 4w 4€†4Bandwidth Reduction:ù4w 4# €.€€‚ÿBandwidth Reduction `$= 4× 4< F‚]€€ç°f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç®f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æxÎÛ/€†"€‰€æyÎÛ/€†"€‰€æzÎÛ/€†"€‰€€ç°f_ɀ‰€€ç­f_ɀ‰€€ç®f_ɀ‰€‚€ ‚€‚†"€y€‚€€€€€€€†"€K€€€‚ÿ Next: Matrix Multiplication Up: Numerical Problems Previous: Solving Linear EquationsBandwidth Reduction    Input description: A graph G=(V,E), representing an matrix M of zero and nonzero elements. RÎw 4)4„ ր¥€€€€€€€†"€2€‚€€€çøIJ€†"€‰€†"€z€€†"€Q€€€€‚ÿProblem description: Which permutation p of the vertices of V minimizes the length of the longest edge when the vertices are ordered on a line, i.e. minimizes ? Discussion: Bandwidth reduction lurks as a hidden but important problem for both graphs and matrices, and it is important to see how it arises so as to properly recognize it. Applied to matrices, it permutes the rows and columns of a sparse matrix so as to minimize the distance b of any nonzero entry from the center diagonal. This is important in solving linear systems, because Gaussian elimination (see Section ) can be performed in on matrices of bandwidth b. This is a big win over the general algorithm if b << n.     b× 4ÂB4+ $€Å€€€€‚ÿBandwidth minimization on graphs arises in more subtle ways. Arranging a set of n circuit components in a line on a circuit board so as to minimize the length of the lo)4ÂB4ù4ngest wire (and hence time delay) is a bandwidth problem, where each vertex of our graph corresponds to a circuit component and there is an edge for every wire linking two components. Alternatively, consider a hypertext application where we must store large objects (say images) on a magnetic tape. From each image there is a set of possible images we can go to next (i.e. the hyperlinks). To minimize the search time, we seek to place linked images near each other on the tape. This is exactly the bandwidth problem. More general formulations, such as rectangular circuit layouts and magnetic disks, inherit the same hardness and classes of heuristics from the linear versions.              Å)4ÒG4K d€‹ €€‚€€€€€€€€€€€€‚ÿUnfortunately, bandwidth minimization is NP-complete. It stays NP-complete even if the input graph is a tree whose maximum vertex degree is 3, which is an unusually strong condition.   Further, there is no known approximation algorithm for bandwidth reduction, even for trees.   Thus our only options are brute-force search or ad hoc heuristics.Fortunately, ad hoc heuristics have been well-studied in the numerical analysis community, and production-quality implementations of the best heuristics are available.   These are based on performing a breadth-first search from a given vertex v, where v is placed at the leftmost point of the ordering.   All of the vertices that are distance 1 from v are placed to its immediate right, followed by all the vertices at distance 2, and so forth until we reach the vertex furthest from v. We then continue the breadth-first search from the vertex immediately to the right of v until all vertices in G are accounted for. The popular heuristics differ according to how many different start vertices are considered and how equidistant vertices are ordered among themselves. However, breaking ties with low-degree vertices over to the left seems to be a good idea.ždÂB4ŠL4T v€Í€€†"€Q‚€€€€ç˜Sa€†"€‰€€‚ÿImplementations of the most popular heuristics, the Cuthill-McKee and Gibbs-Poole-Stockmeyer algorithms, are discussed in the implementation section. The worst case of the Gibbs-Poole-Stockmeyer algorithm is , which would wash out any possible savings in solving linear systems, but its performance in practice is close to linear.    Brute-force search programs can find the exact minimum bandwidth work by backtracking through the set of n! possible permutations of vertices.   Considerable pruning can be achieved to reduce the search space by starting with a good heuristic bandwidth solution and alternately adding vertices to the left- and rightmost open slots in the partial permutation.   The first edge connecting a vertex on the left to a vertex on the right will likely define an edge whose length is greater than our best example to date, thus leading to fast pruning. In our experience, graphs of size n=30 or so can be solved to optimality. See the discussion on backtracking in Section . However, for almost any application such an exact solution will not be worth the expense of finding it. -ŸÒG4·N4Ž ê€A€€æ{ÎÛ/€‰€ç)Ÿ;C€‰€ç݈K€‰€ç%ƒK€‰€ç<K€‰€ç‰K€‰€çw' €†"€‰€砂K€‰€‚ÿImplementations: Fortran language implementations of both the Cuthill-McKee algorithm [CGPS76, Gib76, CM69] and the Gibbs-Poole-Stockmeyer algorithm [Lew82, GPS76] are available from Netlib. See Section .     Empirical evaluations of these and other algorithms on a test suite of 30 matrices are discussed in [Eve79b], showing Gibbs-Poole-Stockmeyer to be the consistent winner.šTŠL4k4T v€«€€€€†"€€‚€çŸ;C€‰€砂K€‰€‚ÿBrute-force implementations written by Ingmar Bitter, Christian Joita, and Dario Vlah in C and C++ as Stony Brook class   projects and capable of solving instances of size n=30 to optimality are provided on the algorithm repository http://ww·N4k4ù4w.cs.sunysb.edu/ algorith.     Notes: An excellent survey on graph-theoretic and algorithmic results on the bandwidth problem to 1981 appears in [CCDG82]. Ad hoc heuristics have been widely studied, a tribute to its importance in numerical computation. Everstine [Eve79b] cites no less than 49 different bandwidth reduction algorithms!Jº·N4µƒ4 î€y€€çl‘K€‰€çZŠK€‰€€€çǗK€‰€矂K€‰€€‚€çøIJ€†"€‰€ç]œ‚€†"€‰€€‚ÿThe hardness of the bandwidth problem was first established by Papadimitriou [Pap76b], and its hardness on trees of maximum degree 3 in [GGJK78].   There are algorithms that run in polynomial time for fixed bandwidth k [Sax80]. An exposition on the hardness of the linear arrangement problem appears in [Eve79a]. Related Problems: Solving linear equations (see page ), topological sorting (see page ).    )k4ރ4& €€€‚ÿÄõƒ4¢…4 Ё—€€‚ç°f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç®f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ|ÎÛ/€†"€‰€æ}ÎÛ/€†"€‰€æâÕÛ/€†"€‰€€ç°f_ɀ‰€€ç­f_ɀ‰€€ç®f_ɀ‰€€‚‚ÿ Next: Matrix Multiplication Up: Numerical Problems Previous: Solving Linear Equations AlgorithmsMon Jun 2 23:33:50 EDT 1997)ރ4˅4& €€€‚ÿµf¢…4€†4O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1˅4±†41Uÿÿÿÿÿÿÿÿ ÿÿÿÿ±†4Ն4$€†4Ն4" €€€ ÿF±†4‡41&ÿÿÿÿÿÿÿÿ‡4W‡4t 5Matrix Multiplication<Ն4W‡4# €2€€‚ÿMatrix Multiplication 2ù‡4‰‰49 @‚ €€ç±f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç¯f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÙÕÛ/€†"€‰€æÚÕÛ/€†"€‰€æÛÕÛ/€†"€‰€€ç±f_ɀ‰€€ç­f_ɀ‰€€ç¯f_ɀ‰€‚€ ‚€‚†"€{€‚€†"€”€€†"€|€€€‚ÿ Next: Determinants and Permanents Up: Numerical Problems Previous: Bandwidth ReductionMatrix Multiplication    Input description: An matrix A and a matrix B. ©gW‡42Œ4B R€Ó€€€†"€}†"€~€‚€‚ÿProblem description: The matrix . Discussion: Although matrix multiplication is an important problem in linear algebra, its main significance for combinatorial algorithms   is its equivalence to a variety of other problems, such as transitive closure and reduction, solving linear systems, and matrix inversion. Thus a faster algorithm for matrix multiplication implies faster algorithms for all of these problems. Matrix multiplication arises in its own right in computing the results of such coordinate transformations as scaling, rotation, and translation for robotics and computer graphics.        ä•‰‰44O l€/€€†"€”€€†"€|€€€€€€‚ÿThe straightforward algorithm to compute the product of matrix A and matrix B runs in O(x y z) time and is tough to beat in practice:’@2Œ4š4R t€‚€€‚€€€€‚€€€€‚†"€‚ÿ for i=1 to x do for j = 1 to z šB4\Á4f š€‹€€€€€€€€€€€€€€‚†"€€†"€†"€‚‚ÿIn two multiplying bandwidth-b matrices, where all nonzero elements of A and B lie within b elements of the main diagonals,   a speedup to O(x b z) is possible, since zero elements will not contribute to the product.Asymptotically faster algorithms for matrix multiplication exist, based on clever divide-and-conquer recurrences. However, these prove difficult to program and require very large matrices to beat the trivial algorithm.   In particular, some empirical results show thš4\Á4Ն4at Strassen's algorithm is unlikely to beat the straightforward algorithm for , and it is less numerically stable to boot. Other studies have been more encouraging, claiming that the crossover point is as low as . Still, I consider it unlikely that you will speed up any serious application by implementing Strassen's algorithm.ÿ¿š4[Å4@ N€…€€†"€”†"€|†"€}‚ÿThere is a better way to save computation when you are multiplying a chain of more than two matrices together.    Recall that multiplying an matrix by a matrix creates an matrix. Thus multiplying a chain of matrices from left to right might create large intermediate matrices, each taking a lot of time to compute. Matrix multiplication is not commutative, but it is associative, so we can parenthesize the chain in whatever manner we deem best without changing the final product.    A standard dynamic programming algorithm can be used to construct the optimal parenthesization. Whether it pays to do this optimization will depend upon whether your matrices are large enough or your chain is multiplied often enough to justify it. Note that we are optimizing over the sizes of the dimensions in the chain, not the actual matrices themselves. If all your matrices are the same dimensions, you are out of luck, since no such optimization is possible.j…\Á4ÅÈ4å ˜€€€€€€€€€€€€€€€€€€€€€€†"€ƒ†"€„€€€€€€€€€€€€€€€€€€€€€€†"€…€€€€€€€€€€€€€‚ÿMatrix multiplication has a particularly interesting interpretation in counting the number of paths between two vertices in a graph.     Let A be the adjacency matrix of a graph G, meaning A[i,j] = 1 if there is an edge between i and j. Otherwise, A[i,j] = 0.   Now consider the square of this matrix, . If , this means that there must be a k such that A[i,k]=A[k,j]=1, so i to k to j is a path of length 2 in G. More generally, counts the number of paths of length exactly k between i and j. This count includes nonsimple paths, where vertices are repeated, such as i to k to i. û‹[Å4ÀË4p ®€€€æÜÕÛ/€‰€†"€Qç$ƒK€‰€€€€€睁K€‰€çž;C€‰€€€‚ÿImplementations: The quick and dirty algorithm will be your best bet unless your matrices are very large. For example, [CLR90] suggests that n>45 before you have a hope of winning. Experimental results suggest that n > 100 is more realistic [CR76], with Bailey [BLS91] finding a crossover point of n=128 for Cray systems. Strassen's algorithm is difficult to implement efficiently because of the data structures required to maintain the array partitions.     That said, an implementation of Strassen's algorithm in Mathematica by Stan Wagon is offered ``without promise of efficiency'' on the algorithm repository WWW site.3ÊÅÈ4óÍ4i  €™€€ç‚K€‰€çw' €†"€‰€‚ç–K€‰€çw' €†"€‰€‚ÿThe linear algebra library of choice is LAPACK, a descendant of LINPACK [DMBS79], which includes several routines for matrix multiplication. These Fortran codes are available from Netlib as discussed in Section .     Algorithm 601 [McN83] of the Collected Algorithms of the ACM is a sparse matrix package written in Fortran that includes routines to multiply any combination of sparse and dense matrices.     See Section for details.ø_ÀË4÷5™ Ç€€çûJA€†"€‰€†"€Q†"€Qçƒ A€†"€‰€€‚€笞K€‰€ç$ƒK€‰€瑐K€‰€筞K€‰€‚ÿXTango (see Section ) is an algorithm animation system for UNIX and X-windows   that includes an animation of the matrix multiplication algorithm.   A C++, implementation of matrix multiplication is embedded in LEDA (see Section ).     Notes: Winograd's algorithm for fast matrix multiplication reduces the number of multiplications by a factor of two óÍ4÷5Ն4over the straightforward algorithm. It is implementable, although the additional bookkeeping required makes it doubtful whether it is a win.   Expositions on Winograd's algorithm [Win68] include [CLR90, Man89, Win80].ÀDóÍ4·5| ƀ€€†"€€çO˜K€‰€ç(ž;C€‰€ç$ƒK€‰€矁K€‰€çǁK€‰€†"€††"€‡‚ÿIn my opinion, the history of theoretical algorithm design began when Strassen published his -time matrix multiplication algorithm.   For the first time, improving an algorithm in the asymptotic sense became a respected goal in its own right. Good expositions on Strassen's algorithm [Str69] include [Baa88, CLR90, Cra94]. Progressive improvements to Strassen's algorithm have gotten progressively less practical. The current best result for matrix multiplication is Coppersmith and Winograd's [CW87] algorithm, while the conjecture is that suffices.Žõ÷5E5™ ó€€çтK€‰€çgK€‰€€€†"€‡‚ç$ƒK€‰€çîå¿é€‰€†"€š†"€ˆ†"€‰碗K€‰€瑐K€‰€‚ÿThe interest in the squares of graphs goes beyond counting paths.   Fleischner [Fle74] proved that the square of any biconnected graph has a Hamiltonian cycle.       See [LS95] for results on finding the square roots of graphs, i.e. finding A given .The problem of Boolean matrix multiplication can be reduced to that of general matrix multiplication [CLR90]. The four-Russians algorithm for Boolean matrix multiplication [ADKF70] uses preprocessing to construct all subsets of rows for fast retreival in performing the actual multiplication, yielding a complexity of .   Additional preprocessing can improve this to [Ryt85]. An exposition on the four-Russians algorithm, including this speedup, appears in [Man89].  cñ·5š5r ²€ç€€ç(ž;C€‰€ç$ƒK€‰€€‚€çøIJ€†"€‰€çUJN€†"€‰€€‚ÿGood expositions of the matrix-chain algorithm include [Baa88, CLR90], where it is a standard example of dynamic programming.   Related Problems: Solving linear equations (see page ), shortest path (see page ).    )E5Ñ5& €€€‚ÿÅÄš5– 5 Ё™€€‚ç±f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç¯f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÝÕÛ/€†"€‰€æÞÕÛ/€†"€‰€æßÕÛ/€†"€‰€€ç±f_ɀ‰€€ç­f_ɀ‰€€ç¯f_ɀ‰€€‚‚ÿ Next: Determinants and Permanents Up: Numerical Problems Previous: Bandwidth Reduction AlgorithmsMon Jun 2 23:33:50 EDT 1997)Ñ5¿ 5& €€€‚ÿµf– 5t 5O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1¿ 5¥ 51Uÿÿÿÿÿÿÿÿÿÿÿÿ¥ 5É 5$t 5É 5" €€€ ÿL¥ 5 51ÿÿÿÿÿÿÿÿ 5W 5=†5Determinants and PermanentsBÉ 5W 5# €>€€‚ÿDeterminants and Permanents 'ý 5~5* "‚€€çÝf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç°f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æàÕÛ/€†"€‰€æáÕÛ/€†"€‰€æ_ÔÛ/€†"€‰€€çÝf_ɀ‰€€ç­f_ɀ‰€€ç°f_ɀ‰€‚€ ‚€‚†"€Š€‚€†"€K€€€‚ÿ Next: Constrained and Unconstrained Optimization Up: Numerical Problems Previous: Matrix MultiplicationDeterminants and Permanents    Input description: An matrix M. AùW 5¿5H ^€ó€€€€€€€€€€€€‚€‚ÿProblem description: What is the determinant |M| or the permanent perm(M) of the matrix m? Discussion: Determinants of matrices provide a clean and useful abstraction in linear   algebra that can used to solve a variety of problems:gô~52B5s Ž€ë€Ȁ€€€€€€€€‚€€¿52B5É 5€€€€€€‚€€ç­;Â€†"€‰€‚ÿu Testing whether a matrix is singular, meaning that the matrix does not have an inverse. A matrix M is singular iff |M| = 0.   u Testing whether a set of d points lies on a plane in fewer than d dimensions.     If so, the system of equations they define is singular, so |M| = 0.u Testing whether a point lies to the left or right of a line or plane.     This problem reduces to testing whether the sign of a determinant is positive or negative, as discussed in Section .ÿ¿51C5= H€‡€Ȁ€ç­;Â€†"€‰€‚ÿu Computing the area or volume of a triangle, tetrahedron, or other simplicial complex. These quantities are a function of the magnitude of the determinant, as discussed in Section .   y2B5MF5£ ÿ€€€€€€†"€‹€€€€‚†"€Œ‚†"€€€†"€‹‚€€€€çøIJ€†"€‰€€€€€†"€Q‚ÿThe determinant of a matrix M is defined as the sum over all n! possible permutations of the n columns of M:where is the number of pairs of elements out of order (called inversions) in permutation .    A direct implementation of this definition yields an O(n!) algorithm, as does the cofactor expansion method we learned in high school.   However, better algorithms are available to evaluate determinants based on LU-decomposition. They are discussed in Section .   The determinant of M is simply the product of the diagonal elements of the LU-decomposition of M, which can be found in time.TÌ1C5¡I5ˆ ހ¥€€€€€€€€€€‚†"€Ž‚‚†"€Q†"€†"€€€€€†"€‘†"€’€‚ÿA closely related function, called the permanent,   arises often in combinatorial problems.   For example, the permanent of the adjacency matrix of a graph G counts the number of perfect matchings in G.    The permanent of a matrix M is defined bydiffering from the determinant only in that all products are positive.Surprisingly, it is NP-hard to compute the permanent, even though the determinant can easily be computed in time.   The fundamental difference is that , while . Fortunately, there are permanent algorithms that prove to be considerably faster than the O(n!) definition, running in time. Thus finding the permanent of a matrix is not out of the realm of possibility. `ÖMF5L5Š ‵€€æVÔÛ/€‰€çw' €†"€‰€†"€Qçƒ A€†"€‰€‚ç?‘K€‰€çÊnuŽ€†"€‰€€‚ÿImplementations: The linear algebra package LINPACK contains a variety of Fortran routines for computing determinants, optimized for different data types and matrix structures. It can be obtained from Netlib, as discussed in Section . A C++ program to compute determinants in time is embedded in LEDA (see Section ).       Nijenhuis and Wilf [NW78] provide an efficient Fortran routine to compute the permanent of a matrix. See Section . Œ ¡I5N5l Š€E€€€炞;C€‰€‚†"€“ç՝;C€‰€çºá'p€†"€‰€ç#ƒK€‰€‚ÿNotes: Cramer's rule reduces the problems of matrix inversion and solving linear systems to that of computing determinants. However, algorithms based on LU-determination are faster. See [BM53] for an exposition on Cramer's rule.  Determinants can be computed in time using fast   matrix multiplication, as shown in [AHU83]. Section discusses such algorithms. A fast algorithm for computing the sign of the determinant,     an important problem for performing robust geometric computations, is due to Clarkson [Cla92].¢qL5;51 0€ã€€ç¥˜K€‰€‚ÿThe problem of computing the permanent was shown to be #P-complete by Valiant [Val79], where #P is the   class of problems solvable on a ``counting'' machine in polynomial time. A counting machine returns the number of distinct solutions to a problem. Counting the number of Hamiltonian cycles in a graph is a #P-comN5;5É 5plete problem that is trivially NP-hard (and presumably harder),   since any count greater than zero proves that the graph is Hamiltonian. Counting problems can be #P-complete even if the corresponding decision problem can be solved in polynomial time, as shown by the permanent and perfect matchings.%‰N5`ƒ5œ €€çíK€‰€†"€‘ç?‘K€‰€祂K€‰€€‚€çøIJ€†"€‰€çg[ր†"€‰€ç­;Â€†"€‰€€‚ÿMinc [Min78] is the definitive work on permanents. A variant of an -time algorithm due to Ryser for computing the permanent is presented in [NW78]. Recently, probabilistic algorithms have been developed for estimating the permanent [FJ95]. Related Problems: Solving linear systems (see page ), matching (see page ), geometric primitives (see page ).      );5‰ƒ5& €€€‚ÿÖÕ`ƒ5_…5 Ё»€€‚çÝf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç°f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æWÔÛ/€†"€‰€æXÔÛ/€†"€‰€æYÔÛ/€†"€‰€€çÝf_ɀ‰€€ç­f_ɀ‰€€ç°f_ɀ‰€€‚‚ÿ Next: Constrained and Unconstrained Optimization Up: Numerical Problems Previous: Matrix Multiplication AlgorithmsMon Jun 2 23:33:50 EDT 1997)‰ƒ5ˆ…5& €€€‚ÿµf_…5=†5O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ˆ…5n†51Uÿÿÿÿÿÿÿÿÿÿÿÿn†5’†5$=†5’†5" €€€ ÿ[*n†5í†51‹0ÿÿÿÿÿÿÿÿí†5>‡5zF6Constrained and Unconstrained OptimizationQ.’†5>‡5# €\€€‚ÿConstrained and Unconstrained Optimization øí†5Z‰5$ ‚€€çÔf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç±f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æZÔÛ/€†"€‰€æ[ÔÛ/€†"€‰€æ\ÔÛ/€†"€‰€€çÔf_ɀ‰€€ç­f_ɀ‰€€ç±f_ɀ‰€‚€ ‚€‚†"€”€‚€†"€•€‚ÿ Next: Linear Programming Up: Numerical Problems Previous: Determinants and PermanentsConstrained and Unconstrained Optimization    Input description: A function . Š_>‡55G \€Á€€€†"€–€€€‚€‚€€‚ÿProblem description: What point maximizes (or minimizes) the function f? Discussion: Most of this book concerns algorithms that optimize one thing or another. This section considers the general problem of optimizing functions where, due to lack of structure or knowledge, we are unable to exploit   the problem-specific algorithms seen elsewhere in this book.Optimization arises whenever there is an objective function that must be tuned for optimal performance.   Suppose we are building a program to identify good stocks to invest in.   We have available certain financial data to analyze, such as the price-earnings ratio, the interest and inflation rates, and the stock price, all as a function of time t. The key question is how much weight we should give to each of these factors, where these weights correspond to coefficents of a formula:ŸhZ‰5Ÿ5V z€Û€€†"€—‚†"€D†"€E†"€˜†"€™‚‚ÿWe seek the numerical values , , , whose stock-goodness function does the best job of evaluating stocks.    Similar issues arise in tuning evaluation functions for game playing programs such as chess.Unconstrained optimization problems also arise in scientific computation.   Physical systems from protein structures to particles naturally seek to minimize their ``energy functions.'' Thus programs that attempt to simulate nature often define energy potential functions for the possible configurations of objects and then take as the ultimate configuration the one that minimizes this potential.   ËŠ5•À5% €M€€‚ÿGlobal optimization problemŸ5•À5’†5s tend to be hard, and there are lots of ways to go about them.   Ask the following questions to steer yourself in the right direction:JŸ5ßÂ5J b€€Ȁ€€€€€ç^ç̀†"€‰€‚ÿu Am I doing constrained or unconstrained optimization? - In unconstrained optimization,   there are no limitations on the values of the parameters other than that they maximize the value of f. Often, however, there are costs or constraints on these parameters. These constraints make certain points illegal, points that might otherwise be the global optimum. Constrained optimization problems typically require mathematical programming approaches like linear programming, discussed in Section .   Ž6•À5mÆ5X ~€o€Ȁ€€†"€š€€€€€€€€€€€€‚ÿu Is the function I am trying to optimize described by a formula or data? - If the function that you seek to optimize is presented as an algebraic formula (such as the minimum of ), the solution is to analytically take its derivative f'(n) and see for which points p' we have f'(p') = 0.   These points are either local maxima or minima, which can be distinguished by taking a second derivative or just plugging back into f and seeing what happens.     Symbolic computation systems such as Mathematica and Maple are fairly     effective at computing such derivatives, although using computer algebra systems effectively is somewhat of a black art. They are definitely worth a try, however, and you can always use them to plot a picture of your function to get a better idea of what you are dealing with.YïßÂ5ÆÊ5j ¢€å€Ȁ€€€€€€€€€€€€†"€›†"€œ†"€€€‚ÿu How expensive is it to compute the function at a given point? - If the function f is not presented as a formula, what to do depends upon what is given. Typically, we have a program or subroutine that evaluates f at a given point, and so can request the value of any given point on demand. By calling this function, we can poke around and try to guess the maxima. Our freedom to search in such a situation depends upon how efficiently we can evaluate f. If f is just a complicated formula, evaluation will be very fast.   But suppose that f represents the effect of the coefficients on the performance of the board evaluation function in a computer chess program, such that is how much a pawn is worth, is how much a bishop is worth, and so forth. To evaluate a set of coefficients as a board evaluator, we must play a bunch of games with it or test it on a library of known positions.   Clearly, this is time-consuming, so we must be frugal in the number of evaluations of f we use.}RmÆ5CÍ5+ $€¥€Ȁ€€‚ÿu How many dimensions do we have? How many do we need? - The difficulty in finding a global maximum increases rapidly with the number of dimensions (or parameters). For this reason, it often pays to reduce the dimension by ignoring some of the parameters. This runs counter to intuition, for the naive programmer is likely to incorporate as many variables as possible into their evaluation function. It is just too hard to tweak such a complicated function. Much better is to start with the 3 to 5 seemingly most important variables and do a good job optimizing the coefficients for these.NÆÊ561 0€;€Ȁ€€€€‚ÿu How smooth is my function? The main difficulty of global optimization is getting trapped in local optima.   Consider the problem of finding the highest point in a mountain range.     If there is only one mountain and it is nicely shaped, we can find the top by just walking in whatever direction is up. However, if there are many false summits or other mountains in the area, it is difficult to convince ourselves whether we are really at the highest point. Smoothness is the property that enables us to quickly find the local optimum from a given point.   We assume smoothness in seeking the peak of the mountain by walking up. If the height atCÍ56’†5 any given point was a completely random function, there would be no way we could find the optimum height short of sampling every single point.i8CÍ561 0€q€€€€€€‚ÿEfficient algorithms for unconstrained global optimization use derivatives and partial derivatives to find local optima, to point out the direction in which moving from the current point does the most to increase or decrease the function. Such derivatives can sometimes be computed analytically, or they can be estimated numerically by taking the difference between values of nearby points. A variety of steepest descent and conjugate gradient methods to find local optima have been developed, similar in many ways to numerical root-finding algorithms.        X%6^63 4€K€€çʘK€‰€‚‚ÿIt is a good idea to try out several different methods on any given optimization problem. For this reason, we recommend experimenting with the implementations below before attempting to implement your own method. Clear descriptions of these algorithms are provided in several numerical algorithms books, in particular [PFTV86].For constrained optimization, finding points that satisfy all the constraints is often the difficult problem. One approach is to use a method for unconstrained optimization, but add a penalty according to how many constraints are violated.     Determining the right penalty function is problem-specific, but it often makes sense to vary the penalties as optimization proceeds. At the end, the penalties should be very high to ensure that all constraints are satisfied.Œð6ê 6œ å€€ç Æ•ù€†"€‰€€‚æ]ÔÛ/€‰€€€绐K€‰€çϗK€‰€çüž;C€‰€çN—K€‰€çw' €†"€‰€€€‚ÿSimulated annealing is a fairly robust and simple approach to constrained optimization, particularly when we are optimizing over combinatorial   structures (permutations, graphs, subsets) instead of continuous functions. Techniques for simulated annealing are described in Section . Implementations: Several of the Collected Algorithms of the ACM are Fortran codes for unconstrained optimization, most notably Algorithm 566 [MGH81], Algorithm 702 [SF92], and Algorithm 734 [Buc94]. Algorithm 744 [Rab95] does unconstrained optimization in Lisp. They are available from Netlib (see Section ).   Also check out the selection at GAMS, the NIST Guide to Available Mathematical Software, at http://gams.nist.gov.    T-^6> 6' €[€€‚‚ÿNEOS (Network-Enabled Optimization System) provides a unique service, the opportunity to solve your problem on computers and software at Argonne National Laboratory, over the WWW.     Linear programming and unconstrained optimization are both supported. This is worth checking out at http://www.mcs.anl.gov/home/otc/Server/ when you need a solution instead of a program.General purpose simulated annealing implementations are available and probably are the best place to start experimenting with this technique for constrained optimization.     Particularly popular is Adaptive Simulated Annealing (ASA), written in C and retrievable via anonymous ftp from ftp.alumni.caltech.edu [131.215.139.234] in the /pub/ingber directory.   To get on the ASA mailing list send e-mail to asa-request@alumni.caltech.edu.ÌNê 6@6~ ʀŸ€€çœK€‰€ç€žÉ €†"€‰€€‚€çʘK€‰€çûž;C€‰€笞;C€‰€çȂK€‰€‚ÿGenocop, by Zbigniew Michalewicz [Mic92], is a genetic algorithm-based program for constrained and unconstrained optimization, written in C.     I tend to be quite skeptical of genetic algorithms (see Section ), but many people find them irresistible. Genocop is available from ftp://ftp.uncc.edu/coe/evol/ for noncommercial purposes. Notes: Steepest-descent methods for unconstrained optimization are discussed in most books on numerical methods, including [PFTV86, BT92]. Unconstrained optimization is the topic of several books, including [Bre73, F> 6@6’†5le80].ä6> 6úB6® *m€€ç ŠK€‰€ç‘K€‰€çž;C€‰€‚ç€‰K€‰€炉K€‰€ç‰K€‰€畑K€‰€眐K€‰€çۈK€‰€ç܈K€‰€ç‰K€‰€€‚ÿSimulated annealing was devised by Kirkpatrick et. al. [KGV83] as a modern variation of the Metropolis algorithm [MRRT53].       Both use Monte Carlo techniques to compute the minimum energy state of a system. Good expositions on simulated annealing include [AK89].Genetic algorithms were developed and popularized by Holland [Hol75, Hol92].   Expositions on genetic algorithms include [Gol89, Koz92, Mic92]. Tabu search [Glo89a, Glo89b, Glo90]   is yet another heuristic search procedure with a devoted following. µa@6¯C6T x€Æ€€€ç^ç̀†"€‰€çï'Uý€†"€‰€€‚ÿRelated Problems: Linear programming (see page ), satisfiability (see page ).    )úB6ØC6& €€€‚ÿÄïC6œE6 Ё—€€‚çÔf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç±f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ^ÔÛ/€†"€‰€æŠÔÛ/€†"€‰€æÔÛ/€†"€‰€€çÔf_ɀ‰€€ç­f_ɀ‰€€ç±f_ɀ‰€€‚‚ÿ Next: Linear Programming Up: Numerical Problems Previous: Determinants and Permanents AlgorithmsMon Jun 2 23:33:50 EDT 1997)ØC6ÅE6& €€€‚ÿµfœE6zF6O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ÅE6«F61Uÿÿÿÿÿÿÿÿÿÿÿÿ«F6ÏF6$zF6ÏF6" €€€ ÿC«F6G61L8ÿÿÿÿÿÿÿÿG6KG6ã 7Linear Programming9ÏF6KG6# €,€€‚ÿLinear Programming “KG6ÞI6H ^‚¯€€çÕf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÝf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ‚ÔÛ/€†"€‰€æƒÔÛ/€†"€‰€æ„ÔÛ/€†"€‰€€çÕf_ɀ‰€€ç­f_ɀ‰€€çÝf_ɀ‰€‚€ ‚€‚†"€ž€‚€€€€€€€†"€Ÿ†"€ç†"€ €‚ÿ Next: Random Number Generation Up: Numerical Problems Previous: Constrained and Unconstrained OptimizationLinear Programming    Input description: A set S of n linear inequalities on m variables , , and a linear optimization function . WKG65K6B R€+€€€€€€€€€€‚€‚ÿProblem description: Which variable assignment X' maximizes the objective function f while satisfying all inequalities S? Discussion: Linear programming is the most important problem in mathematical optimization and operations research.     Applications include:ÕZÞI6 O6{ Āœ€Ȁ€€€‚€€€€€€€†"€6†"€ç€€€€€€†"€6†"€¡‚ÿu Resource allocation -   We seek to invest a given amount of money so as to maximize our return. Our possible options, payoffs, and expenses can usually be expressed as a system of linear inequalities, such that we seek to maximize our possible profit given the various constraints. Very large linear programming problems are routinely solved by airlines and other corporations.   u Approximating the solution of inconsistent equations -   A set of m linear equations on n variables , , is overdetermined if m > n.   Such overdetermined systems are often inconsistent, meaning that no assignment of variables simultaneously solves all the equations. To find the variable assignment that best fits the equations, we can replace each variable by and solve the new system as a linear program, minimizing the sum of the error terms.}R5K6“€6+ $€¥€Ȁ€€‚ÿu Graph algorithms - Many of the standard graph problems described in this book, such as shortest paths, bipartite matching, and network flow, can all be solved as special cases of linear programming. O6“€6ÏF6       Most of the rest, including traveling salesman, set cover, and knapsack, can be solved using integer linear programming.      ò» O6…ƒ67 <€w€€€€€€€€‚ÿThe standard algorithm for linear programming is called the simplex method.   Each constraint in a linear programming problem acts like a knife that carves away a region from the space of possible solutions. We seek the point within the remaining region that maximizes (or minimizes) f(X). By appropriately rotating the solution space, the optimal point can always be made to be the highest point in the region. Since the region (simplex) formed by the intersection of a set of linear constraints is convex, we can find the highest point by starting from any vertex of the region and walking to a higher neighboring vertex. When there is no higher neighbor, we are at the highest point.   œŒ“€6B†61 0€€€€€€€‚ÿWhile the basic simplex algorithm is not too difficult to program, there is a considerable art to producing an efficient implementation capable of solving large linear programs. For example, large programs tend to be sparse (meaning that most inequalities use few variables), so sophisticated data structures must be used.   There are issues of numerical stability and robustness, as well as which neighbor we should walk to next (so called pivoting rules).   Finally, there exist sophisticated interior-point methods, which cut through the interior of the simplex instead of walking along the outside, that beat simplex in many applications.  §€…ƒ6é‡6' €€€‚‚ÿThe bottom line on linear programming is this: you are much better off using an existing LP code than writing your own. Further, you are much better off paying money than surfing the net.   Linear programming is one algorithmic problem of such economic importance that commercial implementations are far superior to free versions.Issues that arise in linear programming include:Þ£B†6lj6; D€G€Ȁ€€€€€€€‚ÿu Do any variables have integrality constraints? -   It is impossible to send 6.54 airplanes from New York to Washington each business day, even if that value maximizes profit according to your model. Such variables often have natural integrality constraints. A linear program is called an integer program when all its variables have integrality constraints, or a mixed integer progam if some of them do.    ß³é‡6Š6, &€g€Ȁ€€‚ÿUnfortunately, it is NP-complete to solve integer or mixed programs to optimality. However, there are techniques for integer programming that work reasonably well in practice.   Cutting plane techniques solves the problem first as a linear program, and then adds extra constraints to enforce integrality around the optimal solution point before solving it again. After a sufficient number of iterations, the optimum point of the resulting linear program matches that of the original integer program. As with most exponential-time algorithms, run times for integer programming depend upon the difficulty of the problem instance and are unpredictable. If they do not make progress quickly, they are unlikely make much progress over longer periods of time. Therefore, if you have multiple implementations available, it may well pay to try the same problem using different codes in the hopes that one can complete in a reasonable amount of time.òÁlj6˜61 0€ƒ€Ȁ€€€€‚ÿu What if my optimization function or constraints are not linear? - In least-squares curve fitting, we seek the line that best approximates a set of points by minimizing the sum of squares of the distance between each point and the line. In formulating this as a mathematical program, the natural objective function is no longer linear, but quadratic. Unfortunately, quadratic programming is NP-complete, even without integer variables.      Ž]Š6XÄ6W |€œ€ȀçÇèN$€†"€‰€‚€€€€€€€€€‚ÿThere are three˜6XÄ6ÏF6 possible courses of action when you must solve a nonlinear program. The best is to see if you can model it in some other way, as is the case with least-squares fitting. The second is to try to track down special codes for quadratic programming, which do exist.     Finally, you can model your problem as a constrained or unconstrained optimization problem and try to solve it with the codes discussed in Section .u What if my model does not match the input format of my LP solver? - Many linear programming implementations accept models only in so-called standard form, where all variables are constrained to be nonnegative, the object function must be minimized, and all constraints must be equalities (instead of inequalities).   Do not fear. There exist standard transformations to map arbitrary LP models into standard form. To convert a maximization problem to a minimization one, simply multiply each coefficient of the objective function by -1. The remaining problems can be solved by adding slack variables to the model.   See any textbook on linear programming for details.Ÿ|˜6È6B R€û€€æ…ÔÛ/€‰€†"€‚€€‚ÿImplementations: A very useful resource on solving linear programs is the USENET frequently asked question (FAQ) list, maintained by John W. Gregory. In particular, it provides a list of available codes with descriptions of experiences.   Check out the plaintext version at ftp://rtfm.mit.edu/pub/usenet/sci.answers/linear-programming-faq or a slicker WWW version at http://www.skypoint.com/ ashbury/linear-programming-faq.html.The noncommercial code of choice appears to be lp_solve,     written in ANSI C by Michel Berkelaar, who has solved problems as large as 30,000 variables and 50,000 constraints. Lp_solve can also handle (smaller) integer and mixed-integer problems. It is available by anonymous ftp from ftp://ftp.es.ele.tue.nl/pub/lp_solve but is not in the public domain. A user community for lp_solve exists, which has ported it to a variety of different platforms.”IXÄ6ªË6K d€“€€‚ç:‘K€‰€ç%˜K€‰€çö—K€‰€‚ÿNEOS (Network-Enabled Optimization System) provides a unique service, an opportunity to solve your problem on computers and software at Argonne National Laboratory via the WWW.   Linear programming and unconstrained optimization are both supported. This is worth checking out at http://www.mcs.anl.gov/home/otc/Server/ if you need an answer instead of a program.If you are serious about solving large linear programs, you likely need a commercial implementation. The book [MW93] provides an overview of commercial linear programming systems, online at http://www.mcs.anl.gov/home/otc/Guide/SoftwareGuide/index.html. Surveys of commercial LP codes appear in [SR95, Sha93] and in the linear programming FAQ. I have heard good things from various people about CPLEX and AMPL, but do your own research before spending money.   AçÈ6ëÍ6Z ‚€Ñ€€‚çéå¿é€‰€窞;C€‰€€€çw' €†"€‰€‚ÿFor low-dimensional linear programming problems, computational geometry algorithms can outperform more general LP codes.   See ftp://icemcfd.com/pub/linprog.a for a C language implementation of Seidel's randomized incremental LP algorithm, by Mike Hohmeyer.  Algorithm 551 [Abd80] and Algorithm 552 [BR80] of the Collected Algorithms of the ACM are simplex-based codes for solving overdetermined systems of linear equations, in Fortran.     See Section for details.yتË6p7¡ µ€€ç˗K€‰€çœgNڀ†"€‰€‚çΗK€‰€ç#ŠGõ€†"€‰€€‚€çH—K€‰€çƒK€‰€çI—K€‰€ç˗K€‰€‚ÿPascal implementations of the revised and dual simplex methods for linear programming, as well as cutting plane and explicit enumeration algorithms for integer programming, are provided in [SDK83].   See Section . These are likely to work only for small problems.Sedgewick [Sed92] provides a bare bones implementation of the simplex algorithm in C++.   SeëÍ6p7ÏF6e Section for details. Notes: Good expositions on the simplex and ellipsoid algorithms for linear programming include [PS82, Chv83].   Expositions on low-dimensional linear programming include [PS85]. For an implementation-oriented exposition on linear and integer programming, with references to experimental work, see [SDK83].¯ZëÍ67U x€µ€€çȁK€‰€ç1ŠK€‰€ç,ŠK€‰€çŠK€‰€‚ÿThe need for optimization via linear programming arose in logistics problems in World War II. The simplex algorithm was invented by George Danzig in 1947 [Dan63]. Klee and Minty [KM72] proved that the simplex algorithm is exponential in worst case, but it is very efficient in practice. Khachian's ellipsoid algorithm [Kha79] proved that linear programming was polynomial in 1979.   Karmarkar's algorithm [Kar84] is an interior-point method that has proven to be both a theoretical and practical improvement of the ellipsoid algorithm, as well as a challenge for the simplex method.Ùp787@ N€³€€çøK€‰€ç]ŠK€‰€€‚ÿLinear programming is P-complete under log-space reductions [DLR79]. This makes it unlikely to have an NC parallel algorithm, where a problem is in NC iff it can be solved on a PRAM in polylogarithmic time using a polynomial number of processors.   Any problem that is P-complete under log-space reduction cannot be in NC unless P=NC.   See [GHR95] for a thorough exposition of the theory of P-completeness, including an extensive list of P-complete problems. Ëw77T x€ò€€€çÇèN$€†"€‰€ç:oó€†"€‰€€‚ÿRelated Problems: Constrained and unconstrained optimization (see page ), network flow (see page ).    )87,7& €€€‚ÿÙØ7 7 ЁÁ€€‚çÕf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÝf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ†ÔÛ/€†"€‰€æ‡ÔÛ/€†"€‰€æˆÔÛ/€†"€‰€€çÕf_ɀ‰€€ç­f_ɀ‰€€çÝf_ɀ‰€€‚‚ÿ Next: Random Number Generation Up: Numerical Problems Previous: Constrained and Unconstrained Optimization AlgorithmsMon Jun 2 23:33:50 EDT 1997),7. 7& €€€‚ÿµf 7ã 7O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1. 7 71Uÿÿÿÿÿÿÿÿÿÿÿÿ 78 7$ã 78 7" €€€ ÿI 7 714:ÿÿÿÿÿÿÿÿ 7À 7RÏ7Random Number Generation?8 7À 7# €8€€‚ÿRandom Number Generation ÷ 7Ò 7 ‚€€çÖf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÔf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ‰ÔÛ/€†"€‰€æµÔÛ/€†"€‰€æ¬ÔÛ/€†"€‰€€çÖf_ɀ‰€€ç­f_ɀ‰€€çÔf_ɀ‰€‚€ ‚€‚†"€¢€‚€€‚ÿ Next: Factoring and Primality Testing Up: Numerical Problems Previous: Linear ProgrammingRandom Number Generation    Input description: Nothing, or perhaps a seed. ­}À 770 .€û€€€€‚€‚ÿProblem description: Generate a sequence of random integers. Discussion: Random number generation forms the foundation behind such standard algorithmic techniques as simulated annealing and Monte Carlo integration. Discrete event simulations, used to model everything from transportation systems to casino poker, all run on streams of random numbers. Initial passwords and cryptographic keys are typically generated randomly. New developments in randomized algorithms for graph and geometric problems are revolutionizing these fields and establishing randomization as one of the fundamental ideas of computer science.           ×Ò 7™A77 <€¯€€ç K€‰€€€‚ÿUnfortunately, generating random numbers is a task that looks a lot easi7™A78 7er than it really is, primarily because it is fundamentally impossible to produce truly random numbers on any deterministic device.   Von Neumann [vN63] said it best: ``Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.''   All we can hope for are pseudorandom numbers, a stream of numbers that appear as if they were generated randomly.C7ÜC7% €=€€‚ÿThere can be serious consequences to using a bad random number generator. For example, the security of an Internet password scheme was recently invalidated with the discovery that its keys were produced using a random number generator of such small period that brute-force search quickly exhausted all possible passwords. The accuracy of simulations is regularly compromised or invalidated by poor random number generation.   Bottom line: This is an area where people shouldn't mess around, but they do. Issues to think about include:    Œ]™A7hE7/ ,€»€Ȁ€€€‚ÿu Should my program use the same ``random'' numbers each time it runs? - A poker game that deals you the exact same hand each time you play quickly loses interest. One common solution is to use the lower-order bits of the machine clock as a seed or source for random numbers, so that each time the program runs it does something different.   _3ÜC7ÇG7, &€g€Ȁ€€‚ÿSuch methods are perhaps adequate for games, but not for serious simulations. There are liable to be periodicities in the distribution of random numbers whenever calls are made in a loop. Also, debugging is seriously complicated by the fact that the results are not repeatable.   If the program crashes, you cannot go back and discover why. One possible compromise is to use a deterministic pseudorandom number generator, but write the current seed to a file between runs. During debugging, this file can be overwritten with a fixed initial value or seed.  €yhE7kI7+ $€ó€Ȁ€€‚ÿu How good is my compiler's built-in random number generator? - If you need uniformly generated random numbers, and you are not going to bet the farm on the accuracy of your simulation, my recommendation is simply to use what your compiler provides.     Your best opportunity to mess it up is with a bad choice of starting seed, so read the manual for its recommendations.™UÇG7L7D V€«€Ȁ€€€€€€瑑K€‰€‚ÿIf you are going to bet the farm on the quality of your simulation, you had better test your random number generator. Be aware that it is very difficult to eyeball the results and decide whether the output is really random. This is because people have very skewed ideas of how random sources should behave and often see patterns that don't really exist. To evaluate a random number generator, several different tests should be used and the statistical significance of the results established. Such tests are implemented in plab and DIEHARD (discussed below) and explained in [Knu81].~8kI7‚M7F Z€s€Ȁ€€€€€€†"€£€€‚ÿu What if I have to implement my own random number generator? - The algorithm of choice is the linear congruential generator.   It is fast, simple, and (if instantiated with the right constants) gives reasonable pseudorandom numbers. The nth random number is a function of the (n-1)st random number:3L7µM7. ,€ €Ȁ†"€€‚ÿs;‚M7(O78 >€{€Ȁ†"€¥†"€Š‚ÿIn theory, linear congruential generators work the same way roulette wheels do.   The long path of the ball around and around the wheel (captured by ) ends in one of a relatively small number of bins, the choice of which is extremely sensitive to the length of the path (captured by the truncation of the ).,ʵM7`7b ’€™€Ȁ€€€€€€†"€§€€†"€š€€€€€€‚ÿA substantial theory has been developed to select the constants a, c, m, and . The period length is largely (O7`78 7a function of the modulus m, which is typically constrained by the word length of the machine. A presumably safe choice for a 32-bit machine would be , a = 1366, c=150889, and m=714025. Don't get creative and change any of the constants, unless you use the theory or run tests to determine the quality of the resulting sequence.Ü¥(O7<ƒ77 <€K€Ȁ€€€€€€‚ÿu What if I don't want large, uniformly distributed random integers? - The linear congruential generator produces a uniformly distributed sequence of large integers, which can be scaled to produce other uniform distributions.   For uniformly distributed real numbers between 0 and 1, use . Note that 1 cannot be realized this way, although 0 can. If you want uniformly distributed integers between l and h, use .'×`7c†7P n€¯€Ȁ€€€€€€€€€€€€€€‚ÿGenerating random numbers according to a given nonuniform distribution can be a tricky business. The most reliable way to do this correctly is the acceptance-rejection method.   Suppose we bound the desired probability distribution function or geometric region to sample from a box and then select a random point p from the box. This point can be selected by p by generating the x and y coordinates independently, at random. If this p is within the distribution, or region, we can return p as selected at random. If p is in the portion of the box outside the region of interest, we throw it away and repeat with another random point. Essentially, we throw darts at random and report those that hit the target.J$<ƒ7­‡7& €I€Ȁ‚ÿThis method is correct, but it can be slow. If the volume of the region of interest is small relative to that of the box, most of our darts will miss the target.   Efficient generators for Gaussian and other special distributions are described in the references and implementations below.Ÿ[c†7LŠ7D V€·€Ȁ€€€€€€€€€€‚ÿBe cautious about inventing your own technique, however, since it can be tricky to obtain the right probability distribution. For example, an incorrect way to select points uniformly from a circle of radius r would be to generate polar coordinates and select an angle from 0 to and a displacement between 0 and r, both uniformly at random.   In such a scheme, half the generated points will lie within r/2 of the radius, when only one-fourth of them should be! This is a substantial enough difference to seriously skew the results, while being subtle enough that it might escape detection.“b­‡7ߌ71 0€Å€Ȁ€€€€‚ÿu How long should I run my Monte Carlo simulation to get the best results? - It makes sense that the longer you run a simulation, the more accurately the results will approximate the limiting distribution, thus increasing accuracy. However, this is only true until you exceed the period, or cycle length, of your random number generator. At that point, your sequence of random numbers repeats itself, and further runs generate no additional information. Check the period length of your generator before you jack up the length of your simulation. You are liable to be very surprised by what you learn.   »ILŠ7š7r ²€™€€æ­ÔÛ/€‰€€€çAK€‰€€€‚ç9.4^€†"€‰€†"€©†"€ª‚ÿImplementations: An excellent WWW page on random number generation and stochastic simulation is available at http://random.mat.sbg.ac.at/others/. It includes pointers to papers and literally dozens of implementations of random number generators. From there are accessible pLab [Lee94] and DIEHARD, systems for testing the quality of random number generators.    The Stanford Graphbase (see Section ) contains a machine-independent random number generator based on the recurrence . With the proper initialization, this generator has a period of at least .    ª/ߌ7PÃ7{ Āc€€ç­ž;C€‰€çù;C€‰€ç;K€‰€€€çw' €†"€‰€‚†"š7PÃ78 7€瀂K€‰€‚ÿAlgorithm 488 [Bre74], Algorithm 599 [AKD83], and Algorithm 712 [Lev92] of the Collected Algorithms of the ACM are Fortran codes for generating random numbers according to several probability distributions, including normal, exponential, and Poisson distributions. They are available from Netlib (see Section ).         Sim++ is a library of routines for implementing discrete event simulations, built by Robert Cubert and Paul Fishwick, of the University of Florida. It contains random number generators for a variety of different distributions, including uniform, exponential, and normal. Check out http://www.cis.ufl.edu/ fishwick/simpack/simpack.html if you need a random number generator to control a simulation. Fishwick's book [Fis95] describes model design using SimPack.     =Èš7Å7u ž€—€€çƒ A€†"€‰€çΗK€‰€ç#ŠGõ€†"€‰€‚çûJA€†"€‰€€‚ÿLEDA (see Section ) provides a comprehensive random source in C++ for generating random bits, integers, and double precision reals.   Sedgewick [Sed92] provides simple implementations of linear and additive congruential generators in C++. See Section for details.XTango (see Section ) is an algorithm animation system for UNIX and X-windows, which includes an animation illustrating the uniformity of random number generation.   }1PÃ7 È7L f€c€€€瑑K€‰€çʘK€‰€çΘK€‰€‚ÿNotes: Knuth [Knu81] has a thorough and interesting discussion of random number generation, which I heartily recommend. He presents the theory behind several methods, including the middle square and shift-register methods we have not described here, as well as a detailed discussion of statistical tests for validating random number generators. Another good source is [PFTV86] - our recommended constants for the linear congruential generator are drawn from here. Comparisons of different random number generators in practice include [PM88].   'ӍÅ71Ë7T v€«€€çq—K€‰€‚†"€«†"€«痐K€‰€€‚ÿTables of random numbers appear in most mathematical handbooks, as relics from the days before there was ready access to computers. Most notable is [RC55], which provides one million random digits.The deep relationship between randomness, information, and compressibility is explored within the theory of Kolmogorov complexity, which measures the complexity of a string by its compressibility. Truly random strings are incompressible. The string of seemingly random digits of cannot be random under this definition, since the entire sequence is defined by any program implementing a series expansion for . Li and Vitáni [LV93] provide a thorough introduction to the theory of Kolmogorov complexity.      RÓ È7ƒÌ7 ̀¯€€€çÇèN$€†"€‰€ç©Ñ-À†"€‰€ç `4"€†"€‰€çkî:€†"€‰€€‚ÿRelated Problems: Constrained and unconstrained optimization (see page ), generating permutations (see page ), generating subsets (see page ), generating partitions (see page ).        )1Ë7¬Ì7& €€€‚ÿÈǃÌ7tÎ7 ЁŸ€€‚çÖf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÔf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ®ÔÛ/€†"€‰€æ¯ÔÛ/€†"€‰€æ°ÔÛ/€†"€‰€€çÖf_ɀ‰€€ç­f_ɀ‰€€çÔf_ɀ‰€€‚‚ÿ Next: Factoring and Primality Testing Up: Numerical Problems Previous: Linear Programming AlgorithmsMon Jun 2 23:33:50 EDT 1997)¬Ì7Î7& €€€‚ÿµftÎ7RÏ7O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Î7ƒÏ71UÿÿÿÿÿÿÿÿÿÿÿÿƒÏ7§Ï7$RÏ7§Ï7" €€€ ÿPƒÏ7 81ò$ÿÿÿÿÿÿÿÿ 8R8žƒ8Factoring and Primality Testing§Ï7 8§Ï7F#§Ï7R8# €F€€‚ÿFactoring and Primality Testing ÷ 8j8! ‚€€ç×f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÕf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ±ÔÛ/€†"€‰€æ²ÔÛ/€†"€‰€æ³ÔÛ/€†"€‰€€ç×f_ɀ‰€€ç­f_ɀ‰€€çÕf_ɀ‰€‚€ ‚€‚†"€¬€‚€€€€‚ÿ Next: Arbitrary-Precision Arithmetic Up: Numerical Problems Previous: Random Number GenerationFactoring and Primality Testing    Input description: An integer n. {*R8å8Q p€W€€€€€€€€‚€瞇*À†"€‰€‚ÿProblem description: Is n a prime number, and if not what are the factors of n? Discussion: The dual problems of factoring integers and testing primality have surprisingly many applications for a problem long suspected of being only of mathematical interest.   The security of the RSA public-key cryptography system (see Section ) is based on the computational intractability of factoring large integers.   As a more modest application, hash table performance   typically improves when the table size is a prime number. To get this benefit, an initialization routine must identify a prime near the desired table size.   Finally, prime numbers are just interesting to play with.   It is no coincidence that programs to generate large primes often reside in the games directory of UNIX systems.ƒj8h 8l Š€5€€€€‚€€€€€€†"€­€€€€†"€®€€†"€¯‚ÿAlthough factoring and primality testing are related problems, algorithmically they are quite different. There exist algorithms that can demonstrate that an integer is composite (i.e. not prime) without actually giving the factors.   To convince yourself of the plausibility of this, note that you can demonstrate the compositeness of any nontrivial integer whose last digit is 0, 2, 4, 5, 6, or 8 without doing the actual division.The simplest algorithm for both of these problems is brute-force trial division.     To factor n, compute the remainder of n/i for all . The prime factorization of n will contain at least one instance of every i such that , unless n is prime. Make sure you handle the multiplicities correctly and account for any primes larger than .“Kå8û 8H ^€™€€€€çÕ•€†"€‰€‚€€‚ÿSuch algorithms can be sped up by using a precomputed table of small primes to avoid testing all possible i. Surprisingly large numbers of primes can be represented in surprisingly little space by using bit vectors (see Section ).   A bit vector of all odd numbers less than 1,000,000 fits in under 64 kilobytes. Even tighter encodings become possible by eliminating all multiples of three and other small primes.Considerably faster factoring algorithms exist, whose correctness depends upon more substantial number theory. The fastest known algorithm, the number field sieve, uses randomness to construct a system of congruences, the solution of which usually gives a factor of the integer.   Integers with as many at 128 digits have been factored using this method, although such feats require enormous amounts of computation.<Õh 878g œ€³€€†"€°€€€€†"€±†"€²€€瞇*À†"€‰€‚ÿRandomized algorithms make it much easier to test whether an integer is prime.    Fermat's little theorem states that for all a, when n is prime. Suppose we pick a random value of and compute the residue of . If this residue is not 1, we have just proven that n cannot be prime. Such randomized primality tests are very efficient. PGP (see Section ) finds 300+ digit primes using hundreds of these tests in minutes, for use as cryptographic keys.  rû 8µA8U x€A€€€€€€†"€³†"€Ž†"€µ€€€‚ÿAlthough the primes are scattered in a seemingly random way throughout the integers, there is some regularity to t78µA8§Ï7heir distribution.   The prime number theorem states that the number of primes less than n, commonly denoted by , is approximately .   Further, there are never large gaps between primes, so in general, one would expect to examine about integers if one wanted to find the first prime larger than n. This distribution and the fast randomized primality test explain how PGP can find such large primes so quickly. ‘`78FE81 0€Á€€æŽÔÛ/€‰€‚ÿImplementations: My first choice for factoring or primality testing applications would be   PARI, a system capable of handling complex number-theoretic problems on integers with up to 300,000 decimal digits, as well as reals, rationals, complex numbers, polynomials, and matrices. It is written mainly in C, with assembly code for inner loops on major architectures, and includes more than 200 special predefined mathematical functions.   PARI can be used as a library, but it also possesses a calculator mode that gives instant access to all the types and functions. The main advantage of PARI is its speed. On a Unix platform, it is between 5 to 100 times faster than Maple or Mathematica, depending on the applications. PARI is available for PC, Amiga, Macintosh, and most Unix platforms by anonymous ftp at ftp://megrez.ceremab.u-bordeaux.fr/pub/pari/.9èµA8H8Q p€Ñ€€€‚€ç؞;C€‰€çïå¿é€‰€ç$ƒK€‰€‚ÿA Mathematica implementation by Ilan Vardi of Lenstra's elliptic curve method of factorization is available in Packages/NumberTheory/FactorIntegerECM.m of the standard Mathematica distribution and MathSource.     It is designed to find prime factors of up to about 18 digits in reasonable time, extending Mathematica's ability to factor all numbers of up to 40 digits. It is faster when factoring the product of small primes. Notes: Bach and Shallit's book [BS96] is the most comprehensive reference on computational number theory,   while Adleman's excellent survey [Adl94a] describes the state of the art, as well as open problems. Good expositions on modern algorithms for factoring and primality testing include [CLR90].ÓgFE8RK8l Š€Ñ€€ç¿K€‰€çM—K€‰€çИK€‰€ç߉K€‰€‚çv˜K€‰€†"€¶‚ÿThe Miller-Rabin [Mil76, Rab80] randomized primality testing algorithm eliminates problems with Carmichael numbers, which are composite integers that always satisfy Fermat's theorem. The best algorithms for integer factorization include the quadratic-sieve [Pom84] and the elliptic-curve methods [HWL87].        Mechanical sieving devices provided the fastest way to factor integers surprisingly far into the computing era. See [SWM95] for a fascinating account of one such device, built during World War I. Hand-cranked, it proved the primality of in fifteen minutes of sieving time.   .ÐH8€M8^ Š€£€€†"€·€€€€çO—K€‰€çՈK€‰€ç ŠK€‰€‚ÿAn important problem in computational complexity theory is whether P = NP co-NP.     The decision problem ``is n a composite number?'' is perhaps the best candidate for a counterexample. By exhibiting the factors of n, it is trivially in NP. It can be shown to be in co-NP, since every prime has a short proof of its primality [Pra75]. However, there is no evidence it is in P.   For more information on complexity classes, see [GJ79, Joh90].ž^RK8*€8@ N€œ€€ç—K€‰€睗K€‰€€‚ÿA group headed by Arjen Lenstra has regularly broken records for general-purpose integer factoring, using an Internet-distributed implementation of the quadratic sieve factoring method.     The June 1993 factorization of RSA-120 took approximately 830 MIP-years of computation. The April 1994 factorization of RSA-129, famous for appearing in the original RSA paper [RSA78], was factored in eight months using over 1,600 computers.   This was particularly noteworthy because in [RSA78] they had originally predicted such a factorization would take 40 quadrillion yea€M8*€8§Ï7rs using 1970s technology. ºf€M8ä€8T x€Ð€€€瞇*À†"€‰€灒'ꀆ"€‰€€‚ÿRelated Problems: Cryptography (see page ), high precision arithmetic (see page ).    )*€8 8& €€€‚ÿÍÌä€8ڂ8 Ё©€€‚ç×f_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÕf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æàÔÛ/€†"€‰€æ×ÔÛ/€†"€‰€æØÔÛ/€†"€‰€€ç×f_ɀ‰€€ç­f_ɀ‰€€çÕf_ɀ‰€€‚‚ÿ Next: Arbitrary-Precision Arithmetic Up: Numerical Problems Previous: Random Number Generation AlgorithmsMon Jun 2 23:33:50 EDT 1997) 8ƒ8& €€€‚ÿµfڂ8žƒ8O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ƒ8éƒ81Uÿÿÿÿÿÿÿÿÿÿÿÿéƒ8 „8$žƒ8 „8" €€€ ÿOéƒ8\„81;6ÿÿÿÿÿÿÿÿ\„8¡„8åH9Arbitrary-Precision ArithmeticE" „8¡„8# €D€€‚ÿArbitrary-Precision Arithmetic ,\„8͆8' ‚€€çØf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÖf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÙÔÛ/€†"€‰€æÚÔÛ/€†"€‰€æÛÔÛ/€†"€‰€€çØf_ɀ‰€€ç­f_ɀ‰€€çÖf_ɀ‰€‚€ ‚€‚†"€ž€‚€€€€€€‚ÿ Next: Knapsack Problem Up: Numerical Problems Previous: Factoring and Primality TestingArbitrary-Precision Arithmetic    Input description: Two very large integers, x and y. ,Û¡„8ùˆ8Q p€¹€€€€€€€†"€”€€€€€‚€‚ÿProblem description: What is x+y, x-y, , and x / y? Discussion: Any programming language whose level rises above basic assembler supports single- and perhaps double-precision integer/real addition, subtraction, multiplication, and division. But what if we wanted to represent the national debt of the United States in pennies? One trillion dollars worth of pennies requires 15 decimal digits, which is far more than can fit into a 32-bit integer.          )ç͆8"‹8B R€Ñ€€€€çڈK€‰€†"€¹‚‚ÿIn other applications much larger integers are needed.     The RSA algorithm for public-key cryptography requires integer keys of at least 100 digits to achieve any level of security, and 1000 digits are recommended.   Experimenting with number-theoretic conjectures for fun or research always requires playing with large numbers. I once solved a minor open problem [GKP89] by performing an exact computation on the integer .What should you do when you need large integers?*ûùˆ8L8/ ,€÷€Ȁ€€€‚ÿu Am I solving a problem instance requiring large integers, or do I have an embedded application? - If you just need the answer to a specific problem with large integers, such as in the number theory application above, you would be well advised to consider using a computer algebra system like Maple or Mathematica.       These use arbitrary-precision arithmetic as a default and use nice Lisp-like programming languages as a front end, together often reducing your problem to a 5 to 10 line program.Ï"‹8]À86 :€Ÿ€Ȁ‚€€€€€‚ÿIf instead you have an embedded application requiring high-precision arithmetic, you would be well advised to use an existing library. In addition to the four basic operations, you are likely to get additional functions for computing things like greatest common divisor in the bargain.   See the implementations below for details.u Do I need high- or arbitrary-precision arithmetic? - Is there an upper bound on how big your integers can get, or do you really need arbitrary-precision, i.e. unbounded. This determines whether you can use a fixed-length array to represent your integers as opposed to a linked-list of digits. The aL8]À8 „8rray is likely to be simpler and will not be a constraint in most applications.›pL8øÁ8+ $€á€Ȁ€€‚ÿu What base should I do arithmetic in? -   It is conceptually simplest to implement your own high-precision arithmetic package in decimal and represent each integer as a string of base-10 digits, at one digit per node.   However, it is far more efficient to use a higher base, ideally the square root of the largest integer supported fully by hardware arithmetic.Ú«]À8ÒÃ8/ ,€Y€Ȁ†"€º‚ÿWhy? The higher the base, the fewer digits we need to represent the number (compare 64 decimal with 1000000 binary). Since hardware addition usually takes one clock cycle independent of the actual numbers, best performance is achieved using the highest base.   The reason for limiting us to is that in performing high-precision multiplication, we will multiply two of these ``digits'' together and need to avoid overflow.ØøÁ8ÚÆ80 .€±€Ȁ‚€€€‚ÿThe only complication of using a larger base is that integers must be converted to and from base-10 for input and output, but the conversion is easily performed once all four high-precision arithmetical operations are supported.  u How low-level are you willing to get for fast computation? Hardware addition is much faster than a subroutine call, so you are going to take a significant hit on speed whenever your package is used where low-precision arithmetic suffices.   High-precision arithmetic is one of few problems in this book where inner loops in assembly language can be the right idea to speed things up.   Finally, using bit-level masking and shift operations instead of arithmetical operations can be a win.[ÒÃ8YÇ8$ €¶€€‚ÿThe algorithm of choice for each of the five basic arithmetic operations is as follows:DóÚÆ8Ê8Q p€ç€Ȁ€€€‚€€€€€€€€€€€‚ÿu Addition - The basic schoolhouse method of lining up the decimal points and then adding the digits from right to left with carries works in time linear in the number of digits.   More sophisticated carry-look-ahead parallel algorithms are available for low-level hardware implementation. Hopefully they are used on your chip for low-precision addition.u Subtraction - Depending upon the sign bits of the numbers, subtraction can be a special case of addition: (A - (-B)) = (A+B). The tricky part of subtraction is performing the borrow.   This can be simplified by always subtracting from the number with the larger absolute value and adjusting the signs afterwards, so we can be certain there will always be something to borrow from.»rYÇ8XÍ8I `€é€Ȁ€€†"€»çC¯4š€†"€‰€‚ÿu Multiplication - The simplest method of repeated addition will take exponential time on large integers, so stay away. The digit-by-digit schoolhouse method is reasonable to program and will work much better, presumably well enough for your application.   On very large integers, Karatsuba's divide-and-conquer algorithm (cited in the notes) wins. Dan Grayson, author of Mathematica's arbitrary-precision arithmetic, found that the switch-over happened at well under 100 digits.   Even faster on very large integers is an algorithm based on Fourier transforms. A discussion of such algorithms appears in Section .ÊŸÊ8"Ï8+ $€?€Ȁ€€‚ÿu Division - Repeated subtraction will take exponential time, so the easiest reasonable algorithm to use is the long-division method you hated in school.   This is a far more complicated algorithm than needed for the other operations, requiring arbitrary-precision multiplication and subtraction as subroutines, as well as trial and error to determine the correct digit to use at each position of the quotient.Š:XÍ8Ô9l Š€}€Ȁ‚€€€†"€Œ€€†"€œ†"€W€€çÚ?'<€†"€‰€‚ÿIn fact, integer division can be reduced to integer multiplication, although in a nontrivial way, so if you are "Ï8Ô9 „8implementing asymptotically fast multiplication, you can reuse that effort in long division. See the references below for details.u Exponentiation - We can compute in the obvious manner using b-1 multiplications, but a better way is to exploit the fact that . By repeatedly squaring the results of our partial product, we can escape using multiplications, a big win when b is large.     See Section for a discussion of this algorithm.>þ"Ï89@ N€€€†"€Ÿ†"€&†"€¿‚ÿHigh- but not arbitrary-precision arithmetic can be conveniently performed using the Chinese remainder theorem and modular arithmetic.     The Chinese remainder theorem states that an integer between 1 and is uniquely determined by its set of residues mod , where each are relatively prime integers. Addition, subtraction, and multiplication (but not division) can be supported using such residue systems, with the advantage that large integers can be manipulated without complicated data structures.HýÔ9Z9K d€ÿ€€†"€À†"€€‚æÜÔÛ/€‰€€‚ÿMany of these algorithms for computations on long integers can be directly applied to computations on polynomials.   See the references for more details. A particularly useful algorithm is Horner's rule for fast polynomial evaluation.   When is blindly evaluated term by term, multiplications will be performed. Much better is observing that , the evaluation of which uses only a linear number of operations. Implementations: All major commercial computer algebra systems incorporate high-precision arithmetic, including Maple, Mathematica, Axiom, and Macsyma.     If you have access to one of these, this is your best option for a quick, nonembedded application. The rest of this section focuses on source code available for embedded applications. ¶Ž9 9( €€€€‚ÿPARI, developed by Henri Cohen and his colleagues in France, is a system capable of handling complex number-theoretic problems on integers with up to 300,000 decimal digits, as well as reals, rationals, complex numbers, polynomials, and matrices.   It is probably the most powerful free software available for number theory. Written mainly in C (with assembly-language code for speed-critical routines), it includes more than 200 special predefined mathematical functions.   PARI can be used as a library, but it possesses also a powerful calculator mode that gives instant access to all the types and functions.   The main advantage of PARI is its speed. On a Unix platform, it runs between 5 to 100 times faster than Maple or Mathematica, depending on the applications. PARI is available for PC, Amiga, Macintosh, and most Unix platforms by anonymous ftp at ftp://megrez.ceremab.u-bordeaux.fr/pub/pari/.$ÁZ94 9c ”€‡€€ç!˜K€‰€€€çw' €†"€‰€‚çƒ A€†"€‰€‚ÿAlgorithm 693 [Smi91] of the Collected Algorithms of the ACM is a Fortran implementation of floating-point, multiple-precision arithmetic. See Section .      An implementation of arbitrary-precision integer and rational arithmetic in C++ is embedded in LEDA (see Section ), including GCD, square roots, and logarithms as well as the basic four operations.       Sparc assembler code is used for certain time-critical functions.6Ÿ 9v@9x Ÿ€€€ç«ž;C€‰€‚çXŠK€‰€çöT9€†"€‰€çΗK€‰€ç#ŠGõ€†"€‰€€‚ÿImplementations in C of a high-precision calculator with all four elementary operations appear in [BR95]. The authors use base 10 for arithmetic and arrays of digits to represent long integers, with short integers as array indices, thus limiting computations to 32,768 digits. The code for these algorithms is printed in the text and available on disk for a modest fee.Bare bones implementations in C of high-precision multiplication and in Pascal of such special functions as logarithm and arctangent appear in [GBY91].   See Section for further details. Sedgewick [Sed92] 4 9v@9 „8provides a bare bones implementation of polynomial arithmetic in C++.   See Section for details. %}4 9›C9š û€€€瑑K€‰€ç؞;C€‰€‚ç“‘K€‰€çԝ;C€‰€瑐K€‰€€€çQ˜K€‰€çԝ;C€‰€瑑K€‰€çԝ;C€‰€瑑K€‰€‚ÿNotes: Knuth [Knu81] is the primary reference on algorithms for all basic arithmetic operations, including implementations of them in the MIX assembly language.   Bach and Shallit [BS96] provide a more recent treatment of computational number theory.  Expositions on the -time divide-and-conquer algorithm for multiplication [KO63] include [AHU74, Man89].   An FFT-based algorithm multiplies two n-bit numbers in time and is due to Schönhage and Strassen [SS71]. Expositions include [AHU74, Knu81]. The reduction between integer division and multiplication is presented in [AHU74, Knu81].}åv@9F9˜ þ€Ï€€çԝ;C€‰€ç$ƒK€‰€ç$ƒK€‰€‚ç$ƒK€‰€瑐K€‰€€‚€ç !‹€†"€‰€瞇*À†"€‰€€‚ÿGood expositions of algorithms for modular arithmetic and the Chinese remainder theorem include [AHU74, CLR90]. A good exposition of circuit-level algorithms for elementary arithmetic algorithms is [CLR90].Euclid's algorithm for computing the greatest common divisor of two numbers is perhaps the oldest interesting algorithm. Expositions include [CLR90, Man89].   Related Problems: Factoring integers (see page ), cryptography (see page ).    )›C9AF9& €€€‚ÿÆÅF9H9 Ё›€€‚çØf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çÖf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÝÔÛ/€†"€‰€æÞÔÛ/€†"€‰€æßÔÛ/€†"€‰€€çØf_ɀ‰€€ç­f_ɀ‰€€çÖf_ɀ‰€€‚‚ÿ Next: Knapsack Problem Up: Numerical Problems Previous: Factoring and Primality Testing AlgorithmsMon Jun 2 23:33:50 EDT 1997)AF90H9& €€€‚ÿµfH9åH9O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b10H9I91UÿÿÿÿÿÿÿÿÿÿÿÿI9:I9$åH9:I9" €€€ ÿAI9{I91ý+ÿÿÿÿÿÿÿÿ{I9²I9l:Knapsack Problem7:I9²I9# €(€€‚ÿKnapsack Problem p.{I9"L9B R‚u€€çÙf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç×f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ ÕÛ/€†"€‰€æÕÛ/€†"€‰€æÕÛ/€†"€‰€€çÙf_ɀ‰€€ç­f_ɀ‰€€ç×f_ɀ‰€‚€ ‚€‚†"€Á€‚€†"€Â€€†"€ †"€€€€‚ÿ Next: Discrete Fourier Transform Up: Numerical Problems Previous: Arbitrary-Precision ArithmeticKnapsack Problem    Input description: A set of items , where item i has size and value . A knapsack capacity C. ‡0²I9©N9W |€g€€€†"€Ã†"€Ä†"€Å€€€‚€€€‚ÿProblem description: Find the subset that maximizes the value of given that ; i.e. all the items fit in a knapsack of size C. Discussion: The knapsack problem arises whenever there is resource allocation with financial constraints. Given a fixed budget, how do you select what things you should buy. Everything has a cost and value, so we seek the most value for a given cost.   The term knapsack problem invokes the image of the backbacker who is constrained by a fixed-size knapsack and so must fill it only with the most useful items.    !ö"L9ց9+ $€í€€€€‚ÿThe typical formulation in practice is the 0/1 knapsack problem,   where each item must be put entirely in the knapsack or not included at all. Objects cannot be broken up arbitrarily, so its not fair taking one can of coke from a six-pack or opening the can to take just a sip. It is this 0/1 pr©N9ց9:I9operty that makes the knapsack problem hard, for a simple greedy algorithm finds the optimal selection whenever we are allowed to subdivide objects arbitrarily.   For each item, we could compute its ``price per pound'', and take as much of the most expensive item until we have it all or the knapsack is full. Repeat with the next most expensive item, until the knapsack is full. Unfortunately, this 0/1 constraint is usually inherent in most applications.b>©N98‚9$ €|€€‚ÿIssues that arise in selecting the best algorithm include:çց9N„9/ ,€Ï€Ȁ€€€‚ÿu Does every item have the same cost/value or the same size? - If each item is worth the same amount to us as any other item, say $1, I maximize my value by taking the greatest number of items. In this case the optimal solution is to sort the items in order of increasing size and insert them into the knapsack in this order until nothing fits.   The problem is solved similarly when each object has the same size but the costs are different. These are the easy cases of knapsack."÷8‚9p†9+ $€ï€Ȁ€€‚ÿu Does each item have the same ``price per pound''? - In this case, our problem is equivalent to ignoring the price and just trying to minimize the amount of empty space left in the knapsack.   Unfortunately, even this restricted version of the problem is NP-complete, and so we cannot expect an efficient algorithm that always solves the problem. Don't lose hope, however, because knapsack proves to be an ``easy'' hard problem, one that can usually be handled with the algorithms described below.ؗN„9H‡9A P€1€Ȁ€€çú…¿J€†"€‰€‚ÿAn important special case of constant ``price-per-pound'' knapsack is the integer partition problem, presented in cartoon   form in Figure .€Kp†9ȇ95 :€˜€Ȁ†"€Æ€€‚ÿ  Figure: Integer partition is a variant of the Knapsack problem  ’QH‡9Z‰9A P€¥€Ȁ€€€€€€†"€Ç‚ÿHere, we seek to partition the elements of S into two sets A and B such that , or alternately make the difference as small as possible. Integer partition can be thought of as bin packing with two equal-sized bins or knapsack with a capacity of half the total weight, so all three problems are closely related and NP-complete.ŒPȇ9Œ9l Š€¥€Ȁ€€‚€€€€€€€€€€€€€€€†"€È†"€É‚ÿThe constant `price-per-pound' knapsack problem is often called the subset sum problem, because given a set of numbers, we seek a subset   that adds up to a specific target number, i.e. the capacity of our knapsack.u Are all the sizes relatively small integers? - When the sizes of the items and the knapsack capacity C are all integers,   there is an efficient dynamic programming algorithm that finds the optimal solution in time O(n C) and O(C) space. Whether this is good for you depends upon how big C is. For 1,000, this might be great, but not for 10,000,000.é Z‰9 À9Ý ˆ+€Ȁ€€€€€€€€€€€€†"€Ê†"€Ë†"€Ì€€€€€€€€†"€Í€€€€€€†"€Î†"€Ì€€€€†"€Ì†"€Ï€€€€†"€Ë‚ÿThe algorithm works as follows: Let S' be a set of items, and let C[i,S'] be true if and only if there is a subset of S' whose size adds up exactly to i. For the empty set, is false for . One by one we add a new item to S' and update the affected values of C[i,S']. Observe that iff either C[i,S'] or is true, since either we use in our subset or we don't. By performing n sweeps through all C elements, one for each , , and updating the array, we identify which sums of sizes can be realized. The knapsack solution is the largest realizable size. In order to reconstruct the winning subset, we must store the name of the item number that turned C[i] from false to true, for each , and then scan backwards through the array.Œ9 À9:I9Ÿ*Œ9«Â9u ž€[€Ȁ€€€€€€†"€Î†"€Ì€€€€‚€€€ç6 œú€†"€‰€‚ÿThe dynamic programming formulation described above ignored the values of the items. To generalize the algorithm, add a field to each element of the array to store the value of the best subset to date summing up to i. We now update not only when C[i] turns from false to true, but when the sum of the cost of plus the cost of is better than the previous cost of C[i].u What if I have multiple knapsacks? -     When there are multiple knapsacks, your problem is probably better thought of as a bin packing problem. See Section .Ñt À9|Æ9] ˆ€ñ€€†"€6€€†"€Ð†"€Ñç^ç̀†"€‰€‚‚ÿWhen the knapsack capacity gets too large for dynamic programming, exact solutions can be found using integer programming or backtracking.     A 0/1 integer variable is used to denote whether item i is present in the optimal subset. We maximize given the constraint that . Algorithms and implementations of integer and linear programming are discussed in Section .When exact solutions prove too costly to compute, heuristics should be used. The simple greedy heuristic inserts items according to the maximum `price per pound' rule, described above.   Often this heuristic solution is close to optimal, but it can be arbitrarily bad depending upon the problem instance. The ``price per pound'' rule can also be used to reduce the size of the problem instance in exhaustive search-based algorithms by eliminating ``cheap but heavy'' objects from future consideration.〫Â9_É9c ”€€€€€†"€Ò†"€Ó†"€Ô€‚æÕÛ/€‰€ç‘K€‰€‚ÿAnother heuristic is based on scaling.   Dynamic programming works well if the capacity of the knapsack is a reasonably small integer, say . But what if we have a problem with capacity ? We scale down the sizes of all items by a factor of , round the size down to an integer, and then use dynamic programming on the scaled items. Scaling works well in practice, especially when the range of sizes of items is not too large. Implementations: Martello and Toth's book [MT90a] comes with a disk of Fortran implementations of a variety of knapsack algorithms.   This is likely the best source of code currently available.„Ý|Æ9ãÌ9§ ¿€€ç‘K€‰€€€çw' €†"€‰€‚ç˗K€‰€çœgNڀ†"€‰€€‚€ç‘K€‰€ç‘K€‰€ç˗K€‰€ç΂K€‰€‚ÿAlgorithm 632 [MT85] of the Collected Algorithms of the ACM is a Fortran code for the 0/1 knapsack problem, with the twist that it supports multiple knapsacks.   See Section .Pascal implementations of several knapsack algorithms, including backtracking and a refined greedy algorithm, are provided in [SDK83]. See Section for details.   Notes: Martello and Toth's book [MT90a] and survey article [MT87] are the standard references on the knapsack problem, including most theoretical and experimental results. An excellent exposition on integer programming approaches to knapsack problems appears in [SDK83]. See [FP75a] for a computational study of algorithms for 0-1 knapsack problems. ¶_É9Ï9j ¢€o€€†"€¡ç։K€‰€ç(ž;C€‰€ç$ƒK€‰€çՈK€‰€瑐K€‰€‚ÿA polynomial-time approximation scheme is an algorithm that approximates the optimal solution of a problem in time polynomial in both its size and the approximation factor .     This very strong condition implies a smooth tradeoff between running time and approximation quality. Good expositions on the polynomial-time approximation scheme [IK75] for knapsack and subset sum includes [Baa88, CLR90, GJ79, Man89].‡ãÌ9–:r ²€/€€çŒK€‰€çȗK€‰€€‚€ç6 œú€†"€‰€ç^ç̀†"€‰€€‚ÿThe first algorithm for generalized public key encryption by Merkle   and Hellman [MH78] was based on the hardness of the knapsack prÏ9–::I9oblem. See [Sch94] for an exposition. Related Problems: Bin packing (see page ), integer programming (see page ).    )Ï9¿:& €€€‚ÿÏΖ:Ž: Ё­€€‚çÙf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€ç×f_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÕÛ/€†"€‰€æÕÛ/€†"€‰€æÕÛ/€†"€‰€€çÙf_ɀ‰€€ç­f_ɀ‰€€ç×f_ɀ‰€€‚‚ÿ Next: Discrete Fourier Transform Up: Numerical Problems Previous: Arbitrary-Precision Arithmetic AlgorithmsMon Jun 2 23:33:50 EDT 1997)¿:·:& €€€‚ÿµfŽ:l:O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1·::1Uÿÿÿÿÿÿÿÿÿÿÿÿ:Á:$l:Á:" €€€ ÿK: :1ž#ÿÿÿÿÿÿÿÿ :M:·:Discrete Fourier TransformAÁ:M:# €<€€‚ÿDiscrete Fourier Transform n5 :»:9 @‚€€çÚf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çØf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÕÛ/€†"€‰€æ ÕÛ/€†"€‰€æ ÕÛ/€†"€‰€€çÚf_ɀ‰€€ç­f_ɀ‰€€çØf_ɀ‰€‚€ ‚€‚†"€Õ€‚€€€†"€:†"€Ö€€€‚ÿ Next: Combinatorial Problems Up: Numerical Problems Previous: Knapsack ProblemDiscrete Fourier Transform    Input description: A sequence of n real or complex values , , sampled at uniform intervals from a function h. t&M:/ :N j€Q€€€€€€€†"€×†"€Ø€‚€‚ÿProblem description: The discrete Fourier transform H of h, , for . Discussion: Although computer scientists tend to be relatively unfamiliar with Fourier transforms, electrical engineers and signal processors eat them for breakfast. Functionally, Fourier transforms provide a way to convert samples of a standard time-series into the ``frequency domain''. This provides a ``dual'' representation of the function, in which certain operations become easier than in the time domain. Applications of Fourier transforms include:         .ÿ»:] :/ ,€ÿ€Ȁ€€€‚ÿu Filtering - Taking the Fourier transform of a function is equivalent to representing it as the sum of sine functions.       By eliminating undesirable high- and/or low-frequency components (i.e. dropping some of the sine functions) and taking an inverse Fourier transform to get us back into the time domain, we can filter an image to remove noise and other artifacts.   For example, the sharp spike in the figure above describes the period of a single sine function that closely models the input data.µ\/ :@:Y €€œ€Ȁ€€‚€€€€€€€€€†"€†"€‚ÿu Image Compression - A smoothed, filtered image contains less information than a noisy image, while retaining a similar appearance. Thus encoding the smoothed image will require fewer bits to represent than the original image.     By eliminating the coefficients of sine functions that contribute relatively little to the image, we can further reduce the size of the image, at little cost in image fidelity.u Convolution and Deconvolution - Fourier transforms can be used to efficiently compute convolutions of two sequences.    A convolution is the pairwise product of elements from two different sequences, such as in multiplying two n-variable polynomials f and g or multiplying two long integers.     Implementing the product directly takes , while suffices using the fast Fourier transform. Another example comes from image processing.   Because a scanner measures the darkness of an image patch instead of a single point, the scanned input is always blurred. A reconstruction of the original signal can be obtained by deconvoluting the input signal with a Gaussian point-spr] :@:Á:ead function.   Ê] :è@:I `€€Ȁ€€€€€€€€€€€€‚ÿu Computing the correlation of functions -   The correlation function of two functions f(t) and g(t) is defined by3@:A:. ,€ €Ȁ†"€Ù‚ÿ:è@:«C:V z€}€Ȁ†"€Ú†"€Û†"€Ü†"€Ý€€€€‚ÿand can be easily computed using Fourier transforms. Note that if the two functions are similar in shape but one is shifted relative to the other (such as and ), the value of will be large at this shift offset .   As an application, suppose that we want to detect whether there are any funny periodicities in our random number generator.   We can generate a large series of random numbers, turn it into a time series (the ith number at time i), and take the Fourier transform of this series. Any funny spikes will correspond to potential periodicities.  êCA:•H:§ —€€€€†"€Þ†"€ß€€†"€à†"€ß‚†"€á‚‚†"€â‚€€€€‚€€€€†"€†"€€€€€‚ÿThe discrete Fourier transform takes as input n   complex numbers , , corresponding to equally spaced points in a time series, and outputs n complex numbers , , each describing a sine function of given frequency. The discrete Fourier transform is defined byand the inverse Fourier transform is defined bywhich enables us move easily between h and H.Since the output of the discrete Fourier transform consists of n numbers, each of which can be computed using a formula on n numbers, they can be computed in time. The fast Fourier transform (FFT) is an algorithm that computes the discrete Fourier transform in . This is arguably the most important algorithm known, as measured by practical impact, for it opened the door to modern image processing. There are several different algorithms that call themselves FFTs, all of which are based on a divide-and-conquer approach. Essentially, the problem of computing the discrete Fourier transform on n points is reduced to computing two transforms on n/2 points each and is then applied recursively      ЫC:žJ:9 @€£€€€€†"€ã‚€‚ÿThe FFT usually assumes that n is a power of two. If this is not the case for your data, you are usually better off padding your data with zeros to create elements rather than hunting for a more general code.Since many image processing systems have strong real-time constraints, FFTs are often implemented in hardware, or at least in assembly language tuned to the particular machine. Be aware of this possibility if the codes below prove too slow.     Ûl•H:yM:o ¬€Ý€€æ6ÕÛ/€‰€çw' €†"€‰€‚çó‚K€‰€€€çw' €†"€‰€‚ÿImplementations: FFTPACK is a package of Fortran subprograms for the fast Fourier transform of periodic and other     symmetric sequences, written by P. Swartzrauber. It includes complex, real, sine, cosine, and quarter-wave transforms. A C language translation of the main routines is also provided.   FFTPACK resides on Netlib (see Section ) at http://www.netlib.org/fftpack.  Algorithm 545 [Fra79] of the Collected Algorithms of the ACM is an implementation of the fast Fourier transform optimizing virtual memory performance and written in Fortran.     See Section for further information.0ŠžJ:©O:Š â€U€€çûJA€†"€‰€‚†"€ãç‘K€‰€çam:7€†"€‰€çΗK€‰€ç#ŠGõ€†"€‰€€‚ÿXTango (see Section ) is an algorithm animation system for UNIX and X-windows, which includes an interesting animation of the fast Fourier transform.  A Pascal implementation of the fast Fourier transform for points appears in [MS91].   For more details, see Section . Sedgewick [Sed92] provides a bare bones implementation of the fast Fourier transform in C++.   See Section for details. QÇyM:‚:Š ‏€€€çО;C€‰€çʘK€‰€çԝ;C€‰€ç(ž;C€‰€ç©O:‚:Á:$ƒK€‰€瑐K€‰€‚ç¡K€‰€çО;C€‰€‚ÿNotes: Brigham [Bri74] is an excellent introduction to Fourier transforms and the FFT and is strongly recommended, as is the exposition in [PFTV86]. Expositions in algorithms texts on the fast Fourier transform include [AHU74, Baa88, CLR90, Man89].Credit for inventing the fast Fourier transform is usually given to Cooley and Tukey [CT65], although it is not completely deserved. See [Bri74] for a complete history.FŒ©O:L„:Š â€}€€ç“‘K€‰€†"€»çԝ;C€‰€瑐K€‰€€€†"€äçQ˜K€‰€çԝ;C€‰€‚çɁK€‰€€‚ÿAn interesting divide-and-conquer algorithm for polynomial multiplication [KO63] does the job in time and is discussed in [AHU74, Man89]. An FFT-based algorithm that multiplies two n-bit numbers in time is due to Schönhage and Strassen [SS71] and is presented in [AHU74].In recent years, wavelets have been proposed to replace Fourier transforms in filtering.   See [Dau92] for an introduction to wavelets. Ÿj‚: …:T x€Ø€€€ç=ù#d€†"€‰€灒'ꀆ"€‰€€‚ÿRelated Problems: Data compression (see page ), high-precision arithmetic (see page ).    )L„:3…:& €€€‚ÿœŒ …:ð†: Ё‰€€‚çÚf_ɀ†"€ ‰€ç­f_ɀ†"€ ‰€çØf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ-ÕÛ/€†"€‰€æ.ÕÛ/€†"€‰€æ/ÕÛ/€†"€‰€€çÚf_ɀ‰€€ç­f_ɀ‰€€çØf_ɀ‰€€‚‚ÿ Next: Combinatorial Problems Up: Numerical Problems Previous: Knapsack Problem AlgorithmsMon Jun 2 23:33:50 EDT 1997)3…:‡:& €€€‚ÿµfð†:·:O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1‡:ÿ‡:1Uÿÿÿÿÿÿÿÿÿÿÿÿÿ‡:#ˆ:$·:#ˆ:" €€€ ÿGÿ‡:jˆ:1‘ÿÿÿÿÿÿÿÿ jˆ:§ˆ:¹Ë:Combinatorial Problems=#ˆ:§ˆ:# €4€€‚ÿCombinatorial Problems ÎÉjˆ:uŒ: ؁£€€çÛf_ɀ†"€ ‰€çû7ۀ†"€ ‰€çÙf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ0ÕÛ/€†"€‰€æ1ÕÛ/€†"€‰€æ2ÕÛ/€†"€‰€€çÛf_ɀ‰€€çû7ۀ‰€€çÙf_ɀ‰€‚€‚€‚ÿ Next: Sorting Up: A Catalog of Algorithmic Previous: Discrete Fourier TransformCombinatorial ProblemsIn this section, we consider several classic algorithmic problems of a purely combinatorial nature. These include sorting and permutation generation, both of which were among the first nonnumerical problems arising on electronic computers. Sorting, searching, and selection can all be classified in terms of operations on a partial order of keys. Sorting can be viewed as identifying or imposing the total order on the keys, while searching and selection involve identifying specific keys based on their position in the total order.      2û§ˆ:§Ž:7 <€÷€€€€€€€€‚ÿThe rest of this section deals with other combinatorial objects, such as permutations, partitions, subsets, calendars, and schedules.    We are particularly interested in algorithms that rank and unrank combinatorial objects, i.e. that map each distinct object to and from a unique integer. Once we have rank and unrank operations, many other tasks become simple, such as generating random objects (pick a random number and unrank) or listing all objects in order (iterate from 1 to n and unrank).îÇuŒ:•:' €€€‚‚ÿWe conclude with the problem of generating graphs. Graph algorithms are more fully presented in subsequent sections.Books on general combinatorial algorithms, in this restricted sense, include:õ§Ž:–Â:f š€!€Ȁ€€€ç?‘K€‰€çÊnuŽ€†"€‰€‚€€€硗K€‰€‚ÿu •:–Â:#ˆ:Nijenhuis and Wilf [NW78] - This book specializes in algorithms for constructing basic combinatorial objects such as permutations, subsets, and partitions. Such algorithms are often very short but hard to locate and usually are surprisingly subtle. Fortran programs for all of the algorithms are provided, as well as a discussion of the theory behind each of them. See Section for details.u Ruskey [Rus97] - On its completion, this manuscript in preparation will become the standard reference on generating combinatorial objects. A preview is available via the WWW at http://www-csc.uvic.ca/home/fruskey/cgi-bin/html/main.html .gø•:ýÄ:o ¬€ñ€Ȁ€€癑K€‰€琑K€‰€‚€€€祗K€‰€‚€€€çs˜K€‰€‚ÿu Knuth [Knu73a, Knu73b] - The standard reference on searching and sorting, with significant material on combinatorial objects such as permutations.u Reingold, Nievergelt, Deo [RND77] - A comprehensive algorithms text with a particularly thorough treatment of combinatorial generation and search.u Stanton and White [SW86] - An undergraduate combinatorics text with algorithms for generating permutations, subsets, and set partitions. It contains relevant programs in Pascal.”B–Â:‘Æ:R r€‡€Ȁ€€ç˜K€‰€€€皌:œ€†"€‰€‚ÿu Skiena [Ski90] - This description of Combinatorica, a library 230 Mathematica functions for generating combinatorial objects and graph theory (see Section ) provides a distinctive view of how different algorithms can fit together.   Its author is uniquely qualified to write a manual on algorithm design.,ýÄ:œÆ:' € €€€‚ÿ)‘Æ:æÆ:& €€€‚ÿ&œÆ: Ç:# €€€‚ÿÂæÆ:ÎÈ:Á P€ȀçÌAf€‰€‚€çoZHŀ‰€‚€çÓh€‰€‚€ç²Ñ-À‰€‚€ç`4"€‰€‚€çtî:€‰€‚€çÕ|Aà€‰€‚€ç6 H?€‰€‚€痙Nž€‰€‚€çø'Uý€‰€‚ÿu Sorting u Searching u Median and Selection u Generating Permutations u Generating Subsets u Generating Partitions u Generating Graphs u Calendrical Calculations u Job Scheduling u Satisfiability& Ç:ôÈ:# €€€‚ÿ)ÎÈ:É:& €€€‚ÿŸœôÈ:ÛÊ: Ё‹€€‚çÛf_ɀ†"€ ‰€çû7ۀ†"€ ‰€çÙf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ3ÕÛ/€†"€‰€æ4ÕÛ/€†"€‰€æ5ÕÛ/€†"€‰€€çÛf_ɀ‰€€çû7ۀ‰€€çÙf_ɀ‰€€‚‚ÿ Next: Sorting Up: A Catalog of Algorithmic Previous: Discrete Fourier Transform AlgorithmsMon Jun 2 23:33:50 EDT 1997)É:Ë:& €€€‚ÿµfÛÊ:¹Ë:O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Ë:êË:1Uÿÿÿÿÿÿÿÿ!ÿÿÿÿêË:Ì:$¹Ë:Ì:" €€€ ÿ8êË:FÌ:1w7ÿÿÿÿÿÿÿÿ"FÌ:tÌ:ÁÀ;Sorting. Ì:tÌ:# €€€‚ÿSorting 9FÌ:­Î:) ‚3€€çÜf_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æaÕÛ/€†"€‰€æXÕÛ/€†"€‰€æYÕÛ/€†"€‰€€çÜf_ɀ‰€€çÚf_ɀ‰€€çÚf_ɀ‰€‚€ ‚€‚†"€å€‚€€€€‚€€‚ÿ Next: Searching Up: Combinatorial Problems Previous: Combinatorial ProblemsSorting    Input description: A set of n items. Problem description: Arrange the items in increasing order. ƁtÌ:;E X€€€€çEkóO€†"€‰€€€‚‚ÿDiscussion: Sorting is the fundamental algorithmic problem in computer science. Learning the different sorting algorithms is like learning scales for a musician.   Sorting is the first step in solving a host of other algorithm problems, as discussed in Section .­Î:;Ì: Indeed, ``when in doubt, sort'' is one of the first rules of algorithm design.Sorting is also used to illustrate the standard paradigms of algorithm design.   The result is that most programmers are familiar with many different sorting algorithms, which sows confusion as to which should be used for a given application. The following criteria can help you decide:э­Î:P;D V€€Ȁ€€€†"€琑K€‰€‚ÿu How many keys will you be sorting? - For small amounts of data (say ), it really doesn't matter much which of the quadratic-time algorithms you use. Insertion sort is faster, simpler, and less likely to be buggy than bubblesort.       Shellsort is much faster than insertion sort, but it involves somewhat trickier programming and looking up good insert sequences in Knuth [Knu73b].Õ;^;9 @€­€Ȁ†"€‚€€€‚ÿIf you have more than 100 items to sort, it is important to use an -time algorithm, like heapsort, quicksort, or mergesort.     If you have more than 1,000,000 items to sort, you probably need an external-memory algorithm that minimizes disk access.   Both types of algorithms are discussed below.u Will there be duplicate keys in the data? -   When all items have distinct keys, the sorted order is completely defined. However, when two items share the same key, something else must determine which one comes first. For many applications it doesn't matter, so any sorting algorithm is equally good.   Often, ties are broken by sorting on a secondary key, like the first name or initial if the family names collide.}HP;Û;5 8€“€Ȁ€€†"€‚ÿOccasionally, ties need to be broken by their initial position in the data set. If the 5th and 27th items of the initial data set share the same key in such a case, the 5th item must be before the 27th in the final order.   A stable sorting algorithm preserves the original ordering in case of ties. Most of the quadratic-time sorting algorithms are stable, while many of the algorithms are not. If it is important that your sort be stable, it is probably better to explicitly use the initial position as a secondary key rather than trust the stability of your implementation.o;^;J ;4 6€y€Ȁ€€†"€‚ÿu What do you know about your data? - In special applications, you can often exploit knowledge about your data to get it sorted faster or more easily. Of course, general sorting is a fast algorithm, so if the time spent sorting is really the bottleneck in your application, you are a fortunate person indeed.y7Û;à ;B R€o€‘€€€€‚€€€€€€€‚ÿu Is the data already partially sorted? If so, algorithms like insertion sort perform better than they otherwise would.u Do you know the distribution of the keys?     If the keys are randomly or uniformly distributed, a bucket or distribution sort makes sense. Throw the keys into bins based on their first letter, and recur until each bin is small enough to sort by brute force.   This is very efficient when the keys get evenly distributed into buckets. However, bucket sort would be bad news sorting names on the mailing list of the ``Smith Society.''ì›J ;¯;Q p€9€‘€€€€†"€‚€€€€€€€€€‚ÿu Are your keys very long or hard to compare?   If your keys are long text strings, it might pay to use a radix or bucket sort instead of a standard comparison sort, because the time of each comparison can get expensive.   A radix sort always takes time linear in the number of characters in the file, instead of times the cost of comparing two keys.u Is the range of possible keys very small?   If you want to sort a subset of n/2 distinct integers, each with a value from 1 to n, the fastest algorithm would be to initialize an n-element bit vector, turn on the bits corresponding to keys, then scan from left to right and report which bits are on.f)à ;!B;= H€S€Ȁ€€€€琑K€‰€‚ÿu Do I have to wor¯;!B;Ì:ry about disk accesses? - In massive sorting problems, it may not be possible to keep all data in memory simultaneously. Such a problem is called external sorting, because one must use an external storage device.     Traditionally, this meant tape drives, and Knuth [Knu73b] describes a variety of intricate algorithms for efficiently merging data from different tapes.   Today, it usually means virtual memory and swapping.   Any sorting algorithm will work with virtual memory, but most will spend all their time swapping..í¯;OD;A P€Ý€ȀçЅـ†"€‰€€€‚ÿThe simplest approach to external sorting loads the data into a B-tree (see Section ) and then does an in-order traversal of the tree to read the keys off in sorted order.   Other approaches are based on mergesort. Files containing portions of the data are sorting using a fast internal sort, and then these files are merged in stages using 2- or k-way merging. Complicated merging patterns based on the properties of the external storage device can be used to optimize performance.ߎ!B;.F;+ $€i€Ȁ€€‚ÿu How much time do you have to write and debug your routine? -     If I had under an hour to deliver a working routine, I would probably just use a simple selection sort.   If I had an afternoon to build an efficient sort routine, I would probably use heapsort, for it delivers reliable performance without tuning. If I was going to take the time required to build a fast system sort routine, I would carefully implement quicksort.ÓOD;=H;< F€©€€çDkóO€†"€‰€‚‚ÿThe best general-purpose sorting algorithm is quicksort (see Section ), although it requires considerable tuning effort to achieve maximum performance.   Indeed, you are probably better off using a library function instead of doing it yourself. A poorly written quicksort will likely run more slowly than a poorly written heapsort.If you are determined to implement your own quicksort, use the following heuristics, which make a big difference in practice:9ë.F;vJ;N j€Ù€Ȁ€€€ç©Ñ-À†"€‰€‚€€€‚ÿu Use randomization -   By randomly permuting (see Section ) the keys before sorting, you can eliminate the potential embarrassment of quadratic-time behavior on nearly-sorted data.u Median of three -   For your pivot element, use the median of the first, last, and middle elements of the array, to increase the likelihood of partitioning the array into roughly equal pieces. Some experiments suggest using a larger sample on big subarrays and a smaller sample on small ones.x:=H;îL;> J€w€Ȁ€€‚€€€†"€W‚ÿu Leave small subarrays for insertion sort -   Terminating the quicksort recursion and switching to insertion sort makes sense when the subarrays get small, say fewer than 20 elements. You should experiment to determine the best switchpoint for your implementation.u Do the smaller partition first -   Assuming that your compiler is smart enough to remove tail recursion, you can minimize runtime memory by processing the smaller partition before the larger one.   Since successive stored calls are at most half as large as before, only stack space is needed.ßYvJ;ÍO;† ڀ·€€çRž;C€‰€€‚æZÕÛ/€‰€ç‘K€‰€çam:7€†"€‰€‚çXŠK€‰€ç#ŠGõ€†"€‰€‚ÿBefore you get started, see Bentley's article on building a faster quicksort [Ben92b]. Implementations: Pascal implementations of all the primary sorting algorithms are available from [MS91]. See Section for details. Timing comparisons show an optimized version of quicksort to be the winner.  Bare bones implementations of all basic sorting algorithms, in C and Pascal, appear in [GBY91].   Most notable is the inclusion of implementations of external memory sorting algorithms. Sedgewick includes similar sort routine fragments in C++.   See Section for details.Ï`îL;š‚;o ¬€Å€€çûJA€†"€ÍO;š‚;Ì:‰€‚ç+Ÿ;C€‰€çىK€‰€€€çw' €†"€‰€‚ÿXTango (see Section ) is an algorithm animation system for UNIX and X-windows, which includes animations of all the basic sorting algorithms, including bubblesort, heapsort, mergesort, quicksort, radix sort, and shellsort. Many of these are quite interesting to watch. Indeed, sorting is the canonical problem for algorithm animation.                Algorithm 410 [Cha71] and Algorithm 505 [Jan76] of the Collected Algorithms of the ACM are Fortran codes for sorting. The latter is an implementation of Shellsort on linked lists. Both are available from Netlib (see Section ).      ÍO;­„;x Ÿ€€€ç«ž;C€‰€‚ç?‘K€‰€çÊnuŽ€†"€‰€ç˜K€‰€皌:œ€†"€‰€€‚ÿC language implementations of Shellsort, quicksort, and heapsort appear in [BR95]. The code for these algorithms is printed in the text and available on disk for a modest fee.A bare bones implementation of heapsort in Fortran from [NW78] can be obtained in Section .     A bare bones implementation of heapsort in Mathematica from [Ski90] can be obtained in Section . d$š‚;‡;@ N€I€€€琑K€‰€çXŠK€‰€‚ÿNotes: Knuth [Knu73b] is the best book that has been written on sorting and indeed is the best book that will ever be written on sorting. It is now almost twenty-five years old, and a revised edition is promised, but it remains fascinating reading. One area that has developed since Knuth is sorting under presortedness measures.   A newer and noteworthy reference on sorting is [GBY91], which includes pointers to algorithms for partially sorted data and includes implementations in C and Pascal for all of the fundamental algorithms.ã^­„;ô‰;… ؀œ€€ç՝;C€‰€ç(ž;C€‰€ç$ƒK€‰€瑐K€‰€ç՝;C€‰€燞K€‰€ç[‰K€‰€ç͗K€‰€‚ÿExpositions on the basic internal sorting algorithms appear in every algorithms text, including [AHU83, Baa88, CLR90, Man89]. Treatments of external sorting are rarer but include [AHU83]. Heapsort was first invented by Williams [Wil64]. Quicksort was invented by Hoare [Hoa62], with careful analysis and implementation by Sedgewick [Sed78].   Von Neumann is credited with having produced the first implementation of mergesort, on the EDVAC in 1945.   See Knuth for a full discussion of the history of sorting, dating back to the days of punched-card tabulating machines.m‡;÷Œ;– ú€ã€€†"€…çšž;C€‰€€€€€ç֝;C€‰€çz—K€‰€‚çù‚K€‰€†"€æ瞘K€‰€†"€V†"€ç€‚ÿSorting has a well-known lower bound under the algebraic decision tree model [BO83].   Determining the exact number of comparisons required for sorting n elements, for small values of n, has generated considerable study. See [Aig88, Raw92] for expositions.This lower-bound does not hold under different models of computation.   Fredman and Willard [FW93] present an algorithm for sorting under a model of computation that permits arithmetic operations on keys.   Under a similar model, Thorup [Tho96] developed a priority queue supporting operations, implying an sorting algorithm. è~ô‰;ߍ;j ¢€€€€çЅـ†"€‰€çfZHŀ†"€‰€ç]œ‚€†"€‰€€‚ÿRelated Problems: Dictionaries (see page ), searching (see page ), topological sorting (see page ).      )÷Œ;Ž;& €€€‚ÿº¹ß;; Ёƒ€€‚çÜf_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ[ÕÛ/€†"€‰€æ\ÕÛ/€†"€‰€æ]ÕÛ/€†"€‰€€çÜf_ɀ‰€€çÚf_ɀ‰€€çÚf_ɀ‰€€‚‚ÿ Next: Searching Up: Combinatorial Problems Previous: Combinatorial Problems AlgorithmsMon Jun 2 23:33:50 EDT 1997)Ž; À;& €€€‚ÿ; À;Ì:µf;ÁÀ;O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1 À;òÀ;1Uÿÿÿÿÿÿÿÿ#ÿÿÿÿòÀ;Á;$ÁÀ;Á;" €€€ ÿ: òÀ;PÁ;1 2ÿÿÿÿÿÿÿÿ$PÁ;€Á;—N<Searching0 Á;€Á;# €€€‚ÿSearching âPÁ;Ã;- (‚×€€çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÛf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ^ÕÛ/€†"€‰€æ_ÕÛ/€†"€‰€æ`ÕÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çÛf_ɀ‰€‚€ ‚€‚†"€è€‚€€€€€€€€‚ÿ Next: Median and Selection Up: Combinatorial Problems Previous: SortingSearching    Input description: A set S of n keys, a query key q. >Ò€Á;ÍÅ;l Š€©€€€€€€€€‚€€€çÇèN$€†"€‰€ç˜Sa€†"€‰€‚ÿProblem description: Where is q in S? Discussion: Searching means different things to different people. Searching for the global maximum or minimum of a function is the problem of unconstrained optimization and is discussed in Section . Chess playing programs search for the best move to make next by using alpha-beta minimax search, which is an exhaustive search of the possible moves using a variation of backtracking (see Section ).      VÃ;#É;< F€7€€çЅـ†"€‰€‚‚ÿHere we consider the simple task of searching for a key in a list or in an array, which is a fundamental problem associated with information retrieval.     Dictionary data structures maintain efficient access to sets of keys under insertion and deletion and are discussed in Section . Typical dictionaries include binary trees and hash tables.We treat searching here as a problem distinct from dictionaries because simpler and more efficient solutions emerge when our primary interest is static searching.   These little data structures can yield large performance improvements when properly employed in an innermost loop.     Also, several of the ideas from list and array searching, such as binary search and self-organization, apply to other problems and justify our attention.   ‚ ÍÅ;¥Ì;y À€€€€€€€€€€€†"€é€€†"€é†"€1€€çÚ?'<€†"€‰€‚ÿThere are two basic approaches to array searching: sequential search and binary search. Both are simple, yet have interesting and subtle variations. In sequential search, we simply start from the front of our list or array of keys and compare each successive item against the key until we find a match or reach the end. In binary search, we start with a sorted array of keys. To search for key q, we compare q to the middle key . If q is before , it must reside in the top half of our set; if not, it must reside in the bottom half of our set. By repeating this process on the correct half, we find the key in a total of comparisons. This is a big win over the n/2 comparisons we expect with sequential search. See Section for more on binary search.Ýž#É;‚Î;% €q€€‚ÿSequential search is the simplest algorithm, and likely to be fastest on up to 10-20 elements. For such tiny problems, forget about binary search. Beyond 50-100 elements, there is no question that binary search will be more efficient than sequential search, even factoring in the cost of the sorting (assuming multiple queries).   Other issues do come into play, however, particularly in identifying the proper variant of the algorithm:ÃQ¥Ì;Q<r ²€¥€Ȁ€€€€€€€‚€€€€€€€€€€€€€€€†"€ç‚ÿu How much time can you spend programming? -   Binary search is a notoriously tricky algorithm to program correctly. It took seventeen years after its invention until the first correct version of binary search was published! Don't be afraid to start from one of‚Î;Q<Á; the implementations described below. Test it completely by writing a driver that searches for every key in the set S as well as between the keys.u Are certain items accessed more often than other ones? -   Certain English words (such as ``the'') are much more likely to occur than others (such as ``defenestrate'').   We can reduce the number of comparisons in a sequential search by putting the most popular words at the top of the list and the least popular ones at the bottom.     Further, nonuniform access is typically the rule, not the exception.   Many real-world distributions, such as word use in English, are more accurately modeled by Zipf's law. Under Zipf's law, the ith most frequently accessed key is selected with probability (i-1)/i times the probability of the (i-1)st most popular key, for all .ÑŸ‚Î;"<2 2€?€Ȁ€€€€‚ÿHowever, preordering the list to exploit a skewed access pattern requires knowing the access pattern in advance. For many applications, it can be difficult to obtain such information.   Far easier are self-organizing lists, where the order of the keys changes in response to the queries.   The simplest and best self-organizing scheme is move-to-front; that is, we move the most recently searched-for key from its current position to the front of the list. Popular keys keep getting boosted to the front, while unsearched-for keys drift towards the back of the list. There is no need to keep track of the frequency of access; just move the keys on demand.   Self-organizing lists also exploit locality of reference, since accesses to a given key are likely to occur in clusters.   Any key will be maintained near the top of the list during a cluster of accesses, even if other keys have proven more popular in the past.ÎqQ<ð <] ˆ€å€Ȁ‚€€€€€€€€€€€€€€€†"€¬‚ÿSelf-organization can extend the useful size range of sequential search. However, you should switch to binary search beyond 50-100 elements.u Is the key close by? - Suppose we know that the target key is to the right of position p, and we think it is close by. Sequential search is fast if we are correct, but we will be punished severely whenever we guess wrong. A better idea is to test repeatedly at larger intervals (p+1, p+2, p+4, p+8, p+16, ) to the right until we find a key to the right of our target.     After this, we have a window containing the target and we can proceed with binary search.è|"<Ø <l Š€ý€Ȁ€€€€†"€ê€€€€‚€€€€€çЅـ†"€‰€‚ÿSuch a one-sided binary search finds the target at position p+l using at most comparisons, so it is faster than binary search when l < < n, yet it can never be much worse.   One-sided binary search is particularly useful in unbounded search problems, such as in numerical root finding.u Is my data structure sitting on external memory? -       Once the number of keys grows too large, as in a CD-ROM telephone directory of all the people in the United States, binary search loses its status as the best search technique. Binary search jumps wildly around the set of keys looking for midpoints to compare, and it becomes very expensive to read in a new page from a secondary storage device for each comparison.   Much better are data structures such as B-trees (see Section ), which cluster the keys into pages so as to minimize the number of disk accesses per search.9ð <@<7 <€€Ȁ€€€€€€‚ÿu Can I guess where the key should be? -   In interpolation search, we exploit our understanding of the distribution of keys to guess where to look next. Interpolation search is probably a more accurate description of how we use a telephone book than binary search. For example, suppose we are searching for Washington, George in a sorted telephone book. We would certainly be safe making our first comparison three-fourths of the way down the list, essentially doing two comparisons for thØ <@<Á;e price of one.$þØ <AB<& €ý€Ȁ‚ÿAlthough interpolation search is an appealing idea, we caution against it for three reasons: First, you have to work very hard to optimize your search algorithm before you can hope for a speedup over binary search. Second, even if you get lucky and beat binary search, it is unlikely to be by enough to have justified the exercise.   Third, your program will be much less robust and efficient when the distribution changes, such as when your application gets put to work on French words instead of English.²F@<óD<l Š€‘€€æŒÕÛ/€‰€‚çƒ A€†"€‰€ç‘K€‰€çam:7€†"€‰€€‚ÿImplementations: The basic sequential and binary search algorithms are simple enough to implement that you should likely do them yourself. Still, the routines described below may be useful as models.Gonnet and Baeza-Yates provides code fragments in C and Pascal for sequential, binary, and interpolation search, as well as for related dictionary structures. LEDA (see Section ) provides a sorted array data type in C++ that supports binary search. Many textbooks include implementations of binary search, including [MS91]. See Section for details.         ÀAB<ÿF<L f€€€€ç‘K€‰€癑K€‰€çXŠK€‰€‚ÿNotes: Mehlhorn and Tsakalidis [MT90b] give a through survey of the state-of-the-art in modern data structures. Knuth [Knu73a] provides a detailed analysis and exposition on all fundamental search algorithms and dictionary data structures but omits such modern data structures as red-black and splay trees. Gonnet and Baeza-Yates [GBY91] provide detailed references and experimental results for a wide variety of search algorithms.zþóD<yI<| ƀ€€ç‘K€‰€€€†"€E‚‚†"€ë‚€€€€†"€Vç؞K€‰€çz—K€‰€‚ÿManber [Man89] provides an interesting discussion of variations of binary search, including one-sided binary search and searching for an index in A where .  In linear interpolation search on an array of sorted numbers, the next position probed is given bywhere q is the query numerical key and S the sorted numerical array. If the keys are drawn independently from a uniform distribution, the expected search time is [YY76]. Expositions on interpolation search include [Raw92].aÑÿF<ÚK< €€†"€ç‘K€‰€çԝ;C€‰€çЅـ†"€‰€€‚€çЅـ†"€‰€çÌAf€†"€‰€€‚ÿNonuniform access patterns can be exploited in binary search trees by structuring them so that popular keys are located near the root, thus minimizing search time.   Dynamic programming can be used to construct such optimal search trees in time [Knu73b]. Expositions include [AHU74].     Splay trees are self-organizing tree structures, as discussed in Section . Related Problems: Dictionaries (see page ), sorting (see page ).    )yI<L<& €€€‚ÿ¶µÚK<¹M< Ё{€€‚çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÛf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æƒÕÛ/€†"€‰€æ„ÕÛ/€†"€‰€æ…ÕÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çÛf_ɀ‰€€‚‚ÿ Next: Median and Selection Up: Combinatorial Problems Previous: Sorting AlgorithmsMon Jun 2 23:33:50 EDT 1997)L<âM<& €€€‚ÿµf¹M<—N<O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1âM<ÈN<1Uÿÿÿÿÿÿÿÿ%ÿÿÿÿÈN<ìN<$—N<ìN<" €€€ ÿEÈN<1O<1ƒÿÿÿÿÿÿÿÿ&1O<lO<ŒÍ<Median and Selection;ìN<lO<# €0€€‚ÿMedian and Selection #ü1O<›<' ‚ €€çÿf_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÜf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ†ÕÛ/€lO<›<ìN<†"€‰€æ‡ÕÛ/€†"€‰€æˆÕÛ/€†"€‰€€çÿf_ɀ‰€€çÚf_ɀ‰€€çÜf_ɀ‰€‚€ ‚€‚†"€ì€‚€€€€€€‚ÿ Next: Generating Permutations Up: Combinatorial Problems Previous: SearchingMedian and Selection    Input description: A set of n numbers or keys, and an integer k. 2ÞlO<̓<T v€œ€€€€€€€€‚€€€ç‰K€‰€€€‚ÿProblem description: Find the key that is smaller than exactly k of the n keys. Discussion: Median finding is an essential problem in statistics, where it provides a more robust notion of average than the mean.       The mean wealth of people who have published research papers on sorting is significantly affected by the presence of one William Gates [GP79],   although his effect on the median wealth is merely to cancel out one starving graduate student.Ü«›<©„<1 0€W€€€€€€‚ÿMedian finding is a special case of the more general selection problem, which asks for the ith element in sorted order.   Selection arises in several applications:|C̓<%‡<9 @€‡€Ȁ€€€‚€€€‚ÿu Filtering outlying elements - In dealing with noisy data samples, it is usually a good idea to throw out the largest and smallest 10% or so of them.       Selection can be used to identify the items defining the tenth and ninetieth percentiles, and the outliers are then filtered out by comparing each item to the two selected elements.u Identifying the most promising candidates -   In a computer chess program, we might quickly evaluate all possible next moves, and then decide to study the top 25% more carefully. Selection followed by filtering is the way to go.Å©„<3ˆ<I `€‹€Ȁ€€€€€€€€€€€€‚ÿu Order statistics -       Particularly interesting special cases of selection include finding the smallest element (i=1), the largest element (i=n), and the median element (i= n/2).ºi%‡<íŠ<Q p€×€€€€€€‚†"€€€€€†"€ç‚ÿThe mean of n numbers can be easily computed in linear time by summing the elements and dividing by n. However, finding the median is a more difficult problem. Algorithms that compute the median can easily be generalized to arbitrary selection.The most elementary median-finding algorithm sorts the items in time and then returns the item sitting the (n/2)nd position.   The good thing is that this gives much more information than just the median, enabling you to select the ith element (for all ) in constant time after the sort. However, there are faster algorithms if all you want is the median.Ó3ˆ< Ž<I `€«€€€€€€€€†"€W†"€‚ÿIn particular, there is an O(n) expected-time algorithm based on quicksort. Select a random element in the data set as a pivot, and use it to partition the data into sets of elements less than and greater than the pivot. From the sizes of these sets, we know the position of the pivot in the total order, and hence whether the median lies to the left or right of the pivot. Now we recur on the appropriate subset until it converges on the median. This takes (on average) iterations, with the cost of each iteration being roughly half that of the previous one. This defines a geometric series that converges to a linear-time algorithm, although if you are very unlucky it takes the same time as quicksort, .   ÞŸíŠ<óÀ<? L€C€€‚€€†"€†"€‚ÿMore complicated algorithms are known that find the median in worst-case linear time. However, the expected-time algorithm will likely win in practice. Just make sure to select random pivots in order to avoid the worst case.Beyond mean and median, a third notion of average is the mode, defined to be the element that occurs the greatest number of times in the data set.   The best way to compute the mode sorts the set in time, whi Ž<óÀ<ìN<ch places all identical elements next to each other. By doing a linear sweep from left to right on this sorted set, we can count the length of the longest run of identical elements and hence compute the mode in a total of time.²C Ž<¥Ã<o ¬€€€†"€Ά"€í€‚æ‰ÕÛ/€‰€€€çXŠK€‰€çöT9€†"€‰€‚ÿIn fact, there is no faster worst-case algorithm possible to compute the mode, since the problem of testing whether there exist two identical elements in a set (called element uniqueness) can be shown to have an lower bound.     Element uniqueness is equivalent to asking if the mode is . Possibilities exist, at least theoretically, for improvements when the mode is large by using fast median computations. Implementations: A bare bones implementation in C of the recursive k-selection algorithm appears in [GBY91].   See Section for further details.  óÀ<EÆ<– ú€€€çûJA€†"€‰€€‚€çZ‰K€‰€çЂK€‰€çԝ;C€‰€ç(ž;C€‰€ç$ƒK€‰€çz—K€‰€çz—K€‰€‚ÿXTango (see Section ) is an algorithm animation system for UNIX and X-windows, which includes an animation of the linear-time selection algorithm.   This animation is a good one. Notes: The linear expected-time algorithm for median and selection is due to Hoare [Hoa61]. Floyd and Rivest [FR75] provide an algorithm that uses fewer comparisons on average. Good expositions on linear-time selection include [AHU74, Baa88, CLR90, Raw92], with [Raw92] being particularly enlightening.QÀ¥Ã<–É<‘ ð€…€€€€€€çUž;C€€†"€€‰€†"€î€€€€ç{ž;C€‰€ç#˜K€‰€€€çH‚K€‰€€€‚ÿA sport of considerable theoretical interest is determining exactly how many comparisons are sufficient to find the median of n items.   The linear-time algorithm of Blum et. al. [BFP 72] proves that suffice, but we want to know what c is. A lower bower bound of 2n comparisons for median finding was given by Bent and John [BJ85]. In 1976, Schönhage, Paterson, and Pippenger [SPP76] presented an algorithm using 3n comparisons. Recently, Dor and Zwick [DZ95] proved that 2.95 n comparisons suffice. These algorithms attempt to minimize the number of element comparisons but not the total number of operations, and hence do not lead to faster algorithms in practice.dòEÆ<úÊ<r ²€é€€ç֝;C€‰€çùK€‰€€‚€ç|þ4.€†"€‰€çÌAf€†"€‰€€‚ÿTight combinatorial bounds for selection problems are presented in [Aig88]. An optimal algorithm for computing the mode is presented in [DM80]. Related Problems: Priority queues (see page ), sorting (see page ).    )–É<#Ë<& €€€‚ÿ»ºúÊ<ÞÌ< Ё…€€‚çÿf_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÜf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æŠÕÛ/€†"€‰€æ‹ÕÛ/€†"€‰€æ·ÕÛ/€†"€‰€€çÿf_ɀ‰€€çÚf_ɀ‰€€çÜf_ɀ‰€€‚‚ÿ Next: Generating Permutations Up: Combinatorial Problems Previous: Searching AlgorithmsMon Jun 2 23:33:50 EDT 1997)#Ë<Í<& €€€‚ÿµfÞÌ<ŒÍ<O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Í<íÍ<1Uÿÿÿÿÿÿÿÿ'ÿÿÿÿíÍ<Î<$ŒÍ<Î<" €€€ ÿHíÍ<YÎ<1 2ÿÿÿÿÿÿÿÿ(YÎ<—Î<č=Generating Permutations>Î<—Î<# €6€€‚ÿGenerating Permutations ãYÎ<§=! ‚Ù€€çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ®ÕÛ/€†"€‰€æ¯ÕÛ/€†"€‰€æ°ÕÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çg_ɀ‰€‚€ ‚€‚†"€ï€‚€€€€‚ÿ Next: Generating S—Î<§=Î<ubsets Up: Combinatorial Problems Previous: Median and SelectionGenerating Permutations    Input description: An integer n. Ĉ—Î<k=< F€€€€€€€‚€€€‚ÿProblem description: Generate (1) all or (2) a random or (3) the next permutation of length n. Discussion: A permutation describes an arrangement, or ordering, of things. Many algorithmic problems in this catalog seek the best way to order a set of objects, including traveling salesman (the least-cost order to   visit n cities), bandwidth (order the vertices of a graph   on a line so as to minimize the length of the longest edge), and graph isomorphism   (order the vertices of one graph so that it is identical to another). Any algorithm for solving such problems exactly must construct a series of permutations along the way.    |§=ç={ Ā€€€€€€€€‚€€€€†"€p†"€ð†"€ñ†"€ò†"€ó†"€ô‚ÿThere are n! permutations of n items, which grows so quickly that you can't expect to generate all permutations for n > 11, since 11! = 39,916,800.   Numbers like these should cool the ardor of anyone interested in exhaustive search and help explain the importance of generating random permutations.Fundamental to any permutation-generation algorithm is a notion of order, the sequence in which the permutations are constructed, from first to last.   The most natural generation order is lexicographic, the order they would appear if they were sorted numerically. Lexicographic order for n=3 is , , , , , and finally . Although lexicographic order is aesthetically pleasing, there is often no particular reason to use it. For example, if you are searching through a collection of files, it does not matter whether the filenames are encountered in sorted order, so long as you search through all of them. Indeed, nonlexicographic orders lead to faster and simpler permutation generation algorithms.°k=ï =X ~€c€€€€€€€€€€€€€€€€†"€õ‚ÿThere are two different paradigms for constructing permutations: ranking/unranking and incremental change methods.     Although the latter are more efficient, ranking and unranking can be applied to solve a much wider class of problems, including the other combinatorial generation problems in this book. The key is to define functions rank and unrank on all permutations p and integers n, m, where |p| = n and .Kç=: =G \€ €Ȁ€€€€€†"€ö†"€÷‚ÿu Rank(p) - What is the position of p in the given generation order?   A typical ranking function is recursive, such as , with . Getting this right means relabeling the elements of the smaller permutation to reflect the deleted first element. Thus3ï =m =. ,€ €Ȁ†"€ø‚ÿó: =ì =Œ æ€ñ€Ȁ€€€€€€€€€€€€€€€†"€ù†"€ú†"€û€€€€†"€ü†"€ý‚ÿu Unrank(m,n) - Which permutation is in position m of the n! permutations of n items?   A typical unranking function finds the number of times (n-1)! goes into m and proceeds recursively. Unrank(2,3) tells us that the first element of the permutation must be `2', since but . Deleting from m leaves the smaller problem Unrank(0,2). The ranking of 0 corresponds to the total order, and the total order on the two remaining elements (since `2' has been used) is , so .s*m =_=I `€U€€€€€€€€€€€€€€‚ÿWhat the actual rank and unrank functions are does not matter as much as the fact that they must be inverses.   In other words, p = Unrank(Rank(p), n) for all permutations p. Once you define ranking and unranking functions for permutations, you can solve a host of related problems:YÆì =ÄA=“ ô€€Ȁ€€€€€€€€€€€€€€€€€€€€€€€€€€€€€‚€€€€€€€‚ÿu Sequenci_=ÄA=Î<ng permutations -   To determine the next permutation that occurs in order after p, we can Rank(p), add 1, and then Unrank(p). Similarly, the permutation right before p in order is Unrank(Rank(p)-1,|p|). Counting through the integers from 0 to n!-1 and unranking them is equivalent to generating all permutations.u Generating random permutations -   Select a random integer from 0 to n!-1 and unrank it.ñ~_=µC=s Ž€€Ȁ€€çÕ•€†"€‰€€€€€€€€€€€çù€ÙY€†"€‰€‚ÿu Keep track of a set of permutations - Suppose we want to construct random permutations and act only when we encounter one we have not seen before.   We can set up a bit vector (see Section )   with n! bits, and set bit i to 1 if permutation Unrank(i,n) has been seen.   A similar technique was employed with k-subsets in the Lotto application of Section .¹^ÄA=nF=[ „€Á€€€€€€灒'ꀆ"€‰€€€€€†"€p‚ÿThe rank/unrank method is best suited for small values of n, since n! quickly exceeds the capacity of machine integers, unless arbitrary-precision arithmetic is available (see Section ).   The incremental change methods work by defining the next and previous operations to transform one permutation into another, typically by swapping two elements. The tricky part is to schedule the swaps so that permutations do not repeat until all of them have been generated. See the output picture above for an ordering of the six permutations of with a single swap between successive permutations.)þµC=—I=+ $€ý€€€€‚ÿIncremental change algorithms for sequencing permutations are tricky, but they are concise enough that they can be expressed in a dozen lines of code. See the implementation section for pointers to code.    Because the incremental change is a single swap, these algorithms can be extremely fast - on average, constant time - which is independent of the size of the permutation! The secret is to represent the permutation using an n-element array to facilitate the swap. In certain applications, only the change between permutations is important. For example, in a brute-force program to search for the optimal tour, the cost of the tour associated with the new permutation will be that of the previous permutation, with the addition and deletion of four edges.ô¬nF=‹L=H ^€[€€€€†"€þ‚€€€€€€‚ÿThroughout this discussion, we have assumed that the items we are permuting are all distinguishable.     However, if there are duplicates (meaning our set is a multiset), you can save considerable time and effort by avoiding identical permutations. For example, there are only ten permutations of , instead of 120.   To avoid duplicates use backtracking and generate the permutations in lexicographic order.Generating random permutations is an important little problem that people stumble across often, and often botch up. The right way is the following two-line, linear-time algorithm. We assume that Random[i,n] generates a random integer between i and n, inclusive.È:—I=SN=Ž ê€u€€‚€€€€€€€€€€‚€€€€€€€€€€€€€€€€€€‚€€€‚ÿ for i=1 to n do a[i] = i; for i=1 to n-1 do swap[ a[i], a[ Random[i,n] ]; That this algorithm generates all permutations uniformly at random is not obvious.   If you think so, explain convincingly why the following algorithm does not generate permutations uniformly:»6‹L==… ؀m€€‚€€€€€€€€€€‚€€€€€€€€€€€€€€€€‚€€‚ÿ for i=1 to n do a[i] = i; for i=1 to n-1 do swap[ a[i], a[ Random[1,n] ]; Such subtleties demonstrate why you must be very careful with random generation algorithms. Indeed, we recommend that you try some reasonably extensive experiments with any random SN==Î<generator before really believing it. For example, generate 10,000 random permutations of length 4 and see whether all 24 of them occur approximately the same number of times.   If you understand how to measure statistical significance, you are in even better shape. ™<SN=³„=] ˆ€}€€æ±ÕÛ/€‰€ç?‘K€‰€çÊnuŽ€†"€‰€‚†"€‚ÿImplementations: The best source on generating combinatorial objects is Nijenhuis and Wilf [NW78], who provide efficient Fortran implementations of algorithms to construct random permutations    and to sequence permutations in minimum-change order.   Also included are routines to extract the cycle structure of a permutation. See Section for details.An exciting WWW site developed by Frank Ruskey of the University of Victoria contains a wealth of material on generating combinatorial objects of different types, including permutations, subsets, partitions, and certain graphs. Specifically, there is an interactive interface that lets you specify which type of objects you would like to construct and quickly returns the objects to you. It is well worth checking this out at http://sue.csc.uvic.ca/ cos/.1с=ä†=` Ž€§€€ç˜K€‰€皌:œ€†"€‰€‚ç9.4^€†"€‰€€‚ÿCombinatorica [Ski90] provides Mathematica implementations of algorithms that construct random permutations and sequence permutations in minimum change and lexicographic orders.     It also provides a backracking routine to construct all distinct permutations of a multiset, and it supports various permutation group operations. See Section .  The Stanford GraphBase (see Section ) contains routines to generate all permutations of a multiset. £³„=ùˆ=r ²€G€€€ç̗K€‰€ç?‘K€‰€祗K€‰€硗K€‰€‚ç݉K€‰€砘K€‰€‚ÿNotes: The primary reference on permutation generation is the survey paper by Sedgewick [Sed77]. Good expositions include [NW78, RND77, Rus97].The fast permutation generation methods make only a single swap between successive permutations. The Johnson-Trotter algorithm [Joh63, Tro62] satisfies an even stronger condition, namely that the two   elements being swapped are always adjacent.|ä†=üŠ=‡ ܀ÿ€€çèK€‰€çèK€‰€€‚€ç¿u,€†"€‰€ç `4"€†"€‰€çkî:€†"€‰€€‚ÿIn the days before ready access to computers, books with tables of random permutations [MO63] were used instead of algorithms.   The swap-based random permutation algorithm presented above was first described in [MO63]. Related Problems: Random number generation (see page ), generating subsets (see page ), generating partitions (see page ).      )ùˆ=%‹=& €€€‚ÿÁÀüŠ=æŒ= Ё‘€€‚çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ²ÕÛ/€†"€‰€æ³ÕÛ/€†"€‰€æŽÕÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çg_ɀ‰€€‚‚ÿ Next: Generating Subsets Up: Combinatorial Problems Previous: Median and Selection AlgorithmsMon Jun 2 23:33:50 EDT 1997)%‹==& €€€‚ÿµfæŒ=č=O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1=õ=1Uÿÿÿÿÿÿÿÿ)ÿÿÿÿõ=Ž=$č=Ž=" €€€ ÿCõ=\Ž=1,ÿÿÿÿÿÿÿÿ*\Ž=•Ž=ëF>Generating Subsets9Ž=•Ž=# €,€€‚ÿGenerating Subsets ä\Ž=ŠÀ=! ‚Û€€çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÿf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æµÕÛ/€†"€‰€æ¶ÕÛ/€†"€‰€æÝÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çÿf_ɀ‰€‚€ ‚€‚†"€ÿ€‚€€€€‚ÿ Next: Generating Par•Ž=ŠÀ=Ž=titions Up: Combinatorial Problems Previous: Generating PermutationsGenerating Subsets    Input description: An integer n. g.•Ž= Ã=9 @€_€€€†"€w€‚€‚ÿProblem description: Generate (1) all or (2) a random or (3) the next subset of the integers . Discussion: A subset describes a selection of objects, where the order among them does not matter. Many of the algorithmic problems in this catalog seek the best subset of a group of things: vertex cover seeks the smallest subset of vertices to touch each edge in a graph; knapsack seeks the most profitable subset of items of bounded total size; and set packing seeks the smallest subset of subsets that together cover each item exactly once.         JØŠÀ=WÆ=r ²€»€€†"€U€€€€€€†"€€€†"€‚†"€ƒ†"€€€‚ÿThere are distinct subsets of an n-element set, including the empty set as well as the set itself. This grows exponentially, but at a considerably smaller rate than the n! permutations of n items. Indeed, since 1,048,576, a brute-force search through all subsets of   20 elements is easily manageable, although by n=30, 1,073,741,824, so you will certainly be pushing things.By definition, the relative order among the elements does not distinguish different subsets. Thus is the same as .   However, it is a very good idea to maintain your subsets in a sorted or canonical order, in order to speed up such operations as testing whether two subsets are identical or making them look right when printed.ö³ Ã=MÇ=C T€k€€ç©Ñ-À†"€‰€†"€U‚ÿAs with permutations (see Section ), the key to subset generation problems is establishing a numerical sequence among all subsets. There are three primary alternatives:ƒWÆ=`Ê= Ȁ€€€†"€p†"€Ÿ†"€ †"€†"€p†"€ü†"€j†"€†"€‚€€€€€‚ÿu Lexicographic order -   Lexicographic order is sorted order, and often the most natural order for generating combinatorial objects. The eight subsets of in lexicographic order are , , , , , , , and . Unfortunately, it is surprisingly difficult to generate subsets in lexicographic order. Unless you have a compelling reason to do so, forget about it.u Gray Code -   A particularly interesting and useful subset sequence is the minimum change order,   wherein adjacent subsets differ by the insertion or deletion of exactly one element. Such an ordering, called a Gray code, appears in the output picture above.aËMÇ=ÁÎ=– ú€›€Ȁ‚€€€€€€€€€€€€€€€€€€€€€€€€€†"€U€€†"€U€€€€‚ÿSubset generation in Gray code order can be very fast, because there is a nice recursive construction to sequence subsets. Further, since only one element changes between subsets, exhaustive search algorithms built on Gray codes can be quite efficient. A set cover program would only have to update the change in coverage by the addition or deletion of one subset. See the implementation section below for Gray code subset generation programs.u Binary counting -   The simplest approach to subset generation problems is based on the observation that any subset S' of S is defined by which of the n=|S| items are in S'. We can represent S' by a binary string of n bits, where bit i is 1 iff the ith element of S is in S'.   This defines a bijection between the binary strings of length n, and the subsets of n items. For n=3, binary counting generates subsets in the following order: {}, {3}, {2}, {2,3}, {1}, {1,3}, {1,2}, {1,2,3}.y,`Ê=F>M h€[€Ȁ†"€€€€€€€€€€€‚ÿThis binary representation is the key to solving all subset generation problems. To generate all subsets in order, simply count from 0 to . For each integer, successively mask off each of the bits and compose a subset of exactly the itemsÁÎ=F>Ž= corresponding to `1' bits. To generate the next or previous subset, increment or decrement the integer by one. Unranking a subset is exactly the masking procedure, while ranking constructs a binary number with 1's corresponding to items in S and then converts this binary number to an integer.        7ðÁÎ=}>G \€ã€Ȁ†"€€€€€€€€€‚ÿTo generate a random subset, you could generate a random integer from 0 to and unrank, although you are probably asking for trouble because any flakiness with how your random number generator rounds things off means that certain subsets can never occur. Therefore, a better approach is simply to flip a coin n times, with the ith flip deciding whether to include element i in the subset.   A coin flip can be robustly simulated by generating a random real or large integer and testing whether it is bigger or smaller than half the range. A Boolean array of n items can thus be used to represent subsets as a sort of premasked integer. The only complication is that you must explicitly handle the carry if you seek to generate all subsets.uQF>ò>$ €¢€€‚ÿGeneration problems for two closely related problems arise often in practice:7ê}>)>M h€Ù€Ȁ€€€€€†"€†"€U€€‚ÿu k-subsets -   Instead of constructing all subsets, we may only be interested in the subsets containing exactly k elements. There are such subsets, which is substantially less than , particularly for small values of k.êrò> >x Ÿ€í€Ȁ€€†"€€€€€€€€€€€‚€€€†"€U†"€S†"€ ‚ÿThe best way to construct all k-subsets is in lexicographic order.   The ranking function is based on the observation that there are k-subsets whose smallest element is f. Using this, it is possible to determine the smallest element in the mth k-subset of n items. We then proceed recursively for subsequent elements of the subset. See the implementations below for details.u Strings -   Generating all subsets is equivalent to generating all strings of true and false. To generate all or random strings on alphabets of size , the same basic techniques apply, except there will be strings in total.ž)>± >… ؀7€€æÝÛ/€‰€ç?‘K€‰€€€çÊnuŽ€†"€‰€ç~ž;C€‰€€€€€çw' €†"€‰€‚ÿImplementations: Nijenhuis and Wilf [NW78] provide efficient Fortran   implementations of algorithms to construct random subsets and to sequence subsets in Gray code and lexicographic order. They also provide routines to construct random k-subsets and sequence them in lexicographic order. See Section for details on ftp-ing these programs. Algorithm 515 [BL77] of the Collected Algorithms of the ACM is another Fortran implementation of lexicographic k-subsets, available from Netlib     (see Section ).s >$>Z ‚€7€€†"€‚ç˜K€‰€€€皌:œ€†"€‰€€‚ÿAn exciting WWW site developed by Frank Ruskey of the University of Victoria contains a wealth of material on generating combinatorial objects of different types, including permutations, subsets, partitions, and certain graphs. Specifically, it provides an interactive interface that lets you specify which type of objects you would like to construct and then returns the objects to you. Check this out at http://sue.csc.uvic.ca/ cos/.Combinatorica [Ski90] provides Mathematica implementations     of algorithms to construct random subsets and to sequence subsets in Gray code, binary, and lexicographic order. They also provide routines to construct random k-subsets and strings, and sequence them lexicographically. See Section for further information on Combinatorica. Ž± >äB>™ 9€€€ç?‘K€‰€祗K€‰€硗K€‰€竞K€‰€ç?‘K€‰€‚ç‰K€‰€€€€€€€†"€U瑐K€‰€‚ÿNotes: The primary expositions on subset generation include [N$>äB>Ž=W78, RND77, Rus97]. Wilf [Wil89] provides an update of [NW78], including a thorough discussion of modern Gray code generation problems.Gray codes were first developed [Gra53] to transmit digital   information in a robust manner over an analog channel.   By assigning the code words in Gray code order, the ith word differs only slightly from the (i+1)st, so minor fluctuations in analog signal strength corrupts only a few bits. Gray codes have a particularly nice correspondence to Hamiltonian cycles on the hypercube.     See any of the references above for details. An exposition on the more general problem of constructing Gray codes for k items (instead of subsets) appears in [Man89].9Ù$>D>` Ž€·€€€€€‚€ç©Ñ-À†"€‰€çkî:€†"€‰€€‚ÿThe popular puzzle Spinout, manufactured by Binary Arts Corporation,   can be solved using Gray codes. Related Problems: Generating permutations (see page ), generating partitions (see page ).    )äB>FD>& €€€‚ÿÇÆD> F> Ё€€‚çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çÿf_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÝÛ/€†"€‰€æÝÛ/€†"€‰€æÝÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çÿf_ɀ‰€€‚‚ÿ Next: Generating Partitions Up: Combinatorial Problems Previous: Generating Permutations AlgorithmsMon Jun 2 23:33:50 EDT 1997)FD>6F>& €€€‚ÿµf F>ëF>O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b16F>G>1Uÿÿÿÿÿÿÿÿ+ÿÿÿÿG>@G>$ëF>@G>" €€€ ÿFG>†G>1,ÿÿÿÿÿÿÿÿ,†G>ÂG>€Ï>Generating Partitions<@G>ÂG># €2€€‚ÿGenerating Partitions k?†G>-K>, &‚‘€€çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÝÛ/€†"€‰€æÝÛ/€†"€‰€æÝÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çg_ɀ‰€‚€ ‚€‚†"€ €‚€€€€‚€€€‚ÿ Next: Generating Graphs Up: Combinatorial Problems Previous: Generating SubsetsGenerating Partitions    Input description: An integer n. Problem description: Generate (1) all or (2) a random or (3) the next integer or set partitions of length n. There are two different types of combinatorial objects denoted by the term ``partition'', namely integer partitions and set partitions.     Although they are quite different beasts, it is a good idea to make both a part of your vocabulary:ȍÂG>õM>; D€€Ȁ€€€€€€€‚ÿu Integer partitions of n are sets of nonzero integers that add up to exactly n. For example, the seven distinct integer partitions of 5 are {5}, {4,1}, {3,2}, {3,1,1}, {2,2,1}, {2,1,1,1}, and {1,1,1,1,1}.   An interesting application I encountered that required the generation of integer partitions was in a simulation of nuclear fission.   When an atom is smashed, the nucleus of protons and neutrons is broken into a set of smaller clusters. The sum of the particles in the set of clusters must equal the original size of the nucleus. As such, the integer partitions of this original size represent all the possible ways to smash the atom.Ï•-K>ÄO>: B€-€Ȁ€€†"€ €€‚ÿu Set partitions divide the elements into nonempty subsets. For example, there are fifteen distinct set partitions of n=4: {1234}, {123,4}, {124,3}, {12,34}, {12,3,4}, {134,2}, {13,24}, {13,2,4}, {14,23}, {1,234}, {1,23,4}, {14,2,3}, {1,24,3}, {1,2,34}, and {1,2,3,4}.     Several of the problems in this catalog return set partitions as results, such as vertex coloring and connected components.LøõM>ƒ>T v€÷€€€€€€€€‚†"€ÄO>ƒ>@G> †"€ †"€‚ÿAlthough the number of integer partitions grows exponentially with n, they do so at a refreshingly slow rate. There are only 627 partitions of n=20, and it is even possible to enumerate all partitions of n=100, since there there are only 190,569,292 of them.The best way to generate all partitions is to construct them in lexicographically decreasing order.   The first partition is itself. The general rule is to subtract 1 from the smallest part that is >1 and then collect all the 1's so as to match the new smallest part >1. For example, the partition after is , since the five 1's left after 3-1=2 becomes the smallest part are best packaged as 2,2,1. When the partition is all 1's, we have completed one trip through all the partitions.ÉÄO>*†>E X€“€€€€‚€€€€€€€€‚ÿThis algorithm is not particularly complicated, but it is sufficiently intricate that you should consider using one of the implementations below. In either case, test it to make sure that you get exactly 627 distinct partitions for n=20.Generating integer partitions uniformly at random is a trickier matter than generating random permutations or subsets. This is because selecting the first (i.e. largest) element of the partition has a dramatic effect on the number of possible partitions that can be generated. Observe that no matter how large n is, there is only one partition of n whose largest part is 1.   The number of partitions of n with largest part at most k is given by the recurrenceÙ\ƒ>‰>} ȀÀ€†"€‚†"€†"€æÝÛ/€‰€‚†"€€€€€‚çü 掀†"€‰€‚ÿwith the two boundary conditions and . This function can be used to select the largest part of your random partition with the correct probabilities and, by proceeding recursively, to eventually construct the entire random partition. Implementations are cited below.  Figure: The Ferrers diagram of a random partition of n=1000  Random partitions tend to have large numbers of fairly small parts,   best visualized by a Ferrers diagram as in Figure . Each row of the diagram corresponds to one part of the partition, with the size of each part represented by that many dots.Sò*†>V‹>a €í€€€€†"€†"€†"€†"€€€€€€€‚ÿSet partitions can be generated using techniques similar to integer partitions.   Each set partition is encoded as a restricted growth function, , where and , for .   Each distinct digit identifies a subset, or block, of the partition, while the growth condition ensures that the blocks are sorted into a canonical order based on the smallest element in each block. For example, the restricted growth function 0,1,1,2,0,3,1 defines the set partition {{1,5}, {2,3,7}, {4}, {6} }.Z ‰>°Ž>N j€€€€€‚€€€€€€€€€€€‚ÿSince there is a one-to-one equivalence between set partitions and restricted growth functions, we can use lexicographic order on the restricted growth functions to order the set partitions. Indeed, the fifteen partitions of {1,2,3,4} listed above are sequenced according to the lexicographic order of their restricted growth function (check it out).To randomly generate set partitions, we use a similar counting strategy as with integer partitions. The Stirling numbers of the second kind S(n,k) count the number of partitions of 1,...,n with exactly k blocks. They are computed using the recurrence S(n,k) = S(n-1,k-1) + k*S(n-1,k) with the boundary conditions S(n,n)=1. The reader is referred to the references and implementations for more details and code. ÉuV‹>…Á>T v€í€€æÝÛ/€‰€ç?‘K€‰€çÊnuŽ€†"€‰€‚‚ÿImplementations: The best source on generating combinatorial objects is Nijenhuis and Wilf [NW78], who provide efficient Fortran implementations of algorithms to construct random and sequential integer partitions, set partitions, compositions°Ž>…Á>@G>, and Young tableaux. See Section for details on ftp-ing these programs.An exciting WWW site developed by Frank Ruskey of the University of Victoria contains a wealth of material on generating combinatorial objects of different types, including permutations, subsets, partitions, and certain graphs. It is well worth checking this out at http://sue.csc.uvic.ca/~cos/.±$°Ž>6Ä> è€O€€ç˜K€‰€皌:œ€†"€‰€‚ç¢K€‰€€€€€çw' €†"€‰€ç9.4^€†"€‰€€‚ÿCombinatorica [Ski90] provides Mathematica implementations of algorithms to construct random and sequential integer partitions, compositions, strings, and Young tableaux, as well as to count and manipulate these objects. See Section .Algorithm 403 [BL77] of the Collected Algorithms of the ACM is a Fortran code for constructing integer partitions with k parts. It is available from Netlib     (see Section ). The Stanford GraphBase (see Section ) also contains generators for constructing all integer partitions. Û^…Á>Ç>} Ȁœ€€€ç?‘K€‰€祗K€‰€硗K€‰€‚€€‚ç?‘K€‰€祗K€‰€硗K€‰€€‚ÿNotes: The standard references on combinatorial generation [NW78, RND77, Rus97] all present algorithms for generating integer and/or set partitions. Andrews is the primary reference on integer partitions and related topics.Interestingly, the set of all 52 set partitions for n=5 appears in the form of Murasaki diagrams in the oldest novel known, The Tale of Genji.Two related combinatorial objects are Young tableaux and integer compositions, although they are less likely to emerge in applications. Generation algorithms for both are presented in [NW78, RND77, Rus97] Õ6Ä>É>. *€«€€€€€‚ÿYoung tableaux are two-dimensional configurations of integers {1,...,n} where the number of elements in each row is defined by an integer partition of n. Further, the elements of each row and column are sorted in increasing order, and the rows are left-justified. This notion of shape captures a wide variety of structures as special cases. They have many interesting properties, including the existance of a bijection between pairs of tableaux and permutations.é£Ç>ýË>F Z€G€€€€€€€€€€€€€‚ÿCompositions represent the set of possible assignments of a set of n indistinguishable balls to k distinguishable boxes. For example, we can place three balls into two boxes as {3,0}, {2,1}, {1,2}, or {0,3}. Compositions are most easily constructed sequentially in lexicographic order. To construct them randomly, pick a random (k-1)-subset of n+k-1 items using the algorithm of the previous section, and count the number of unselected items between the selected ones. For example, if k=5 and n=10, the (5-1)-subset {1,3,7,14} of 1,..., (n+k-1)=14 defines the composition {0,1,3,6,0}, since there are no items to the left of element 1 nor right of element 14. ŸjÉ>»Ì>T x€Ø€€€ç©Ñ-À†"€‰€ç `4"€†"€‰€€‚ÿRelated Problems: Generating permutations (see page ), generating subsets (see page ).    )ýË>äÌ>& €€€‚ÿŸœ»Ì>¢Î> Ё‹€€‚çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ˜ÛÛ/€†"€‰€æÛÛ/€†"€‰€æÛÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çg_ɀ‰€€‚‚ÿ Next: Generating Graphs Up: Combinatorial Problems Previous: Generating Subsets AlgorithmsMon Jun 2 23:33:50 EDT 1997)äÌ>ËÎ>& €€€‚ÿµf¢Î>€Ï>O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1ËÎ>±Ï>1Uÿÿÿÿÿÿÿÿ-ÿÿÿÿ±Ï> ?$€Ï> ?" €€€ ÿ±Ï> ?€Ï>B±Ï>N?1 7ÿÿÿÿÿÿÿÿ.N?†?ÑÁ?Generating Graphs8 ?†?# €*€€‚ÿGenerating Graphs „WN? ?- (‚Á€€çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ‘ÛÛ/€†"€‰€æ’ÛÛ/€†"€‰€æ“ÛÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çg_ɀ‰€‚€ ‚€‚†"€€‚€€€€€€€€‚ÿ Next: Calendrical Calculations Up: Combinatorial Problems Previous: Generating PartitionsGenerating Graphs    Input description: Parameters describing the desired graph, such as the number of vertices n, the number of edges m, or the edge probability p. ªz†?Ž?0 .€õ€€€€‚€‚ÿProblem description: Generate (1) all or (2) a random or (3) the next graph satisfying the parameters. Discussion: Graph generation typically arises in constructing test data for programs.   Perhaps you have two different programs that solve the same problem, and you want to see which one is faster or make sure that they always give the same answer.   Another application is experimental graph theory, verifying whether a particular property is true for all graphs or how often it is true.   It is much easier to conjecture the four-color theorem once you have demonstrated 4-colorings for all planar graphs on 15 vertices.E ?ù?' €=€€‚‚ÿA different application of graph generation arises in network design.   Suppose you need to design a network linking ten machines using as few cables as possible, such that the network can survive up to two vertex failures. One approach is to test all the networks with a given number of edges until you find one that will work. For larger graphs, more heuristic approaches, like simulated annealing, will likely be necessary.Many factors complicate the problem of generating graphs. First, make sure you know what you want to generate:a&Ž?Z ?; D€M€Ȁ€€€€€€€‚ÿu Do I want labeled or unlabeled graphs? -     The issue here is whether the names of the vertices matter in deciding whether two graphs are the same. In generating labeled graphs, we seek to construct all possible labelings of all possible graph topologies. In generating unlabeled graphs, we seek only one representative for each topology and ignore labelings. For example, there are only two connected unlabeled graphs on three vertices - a triangle and a simple path. However, there are four connected labeled graphs on three vertices - one triangle and three 3-vertex paths, each distinguished by their central vertex. In general, labeled graphs are much easier to generate. However, there are so many more of them that you quickly get swamped with isomorphic copies of the same few graphs.Zÿù?Ž ?[ „€ÿ€Ȁ€€€€€€€€€€€€€€€€€€‚ÿu What do I mean by random? -   There are two primary models of random graphs, both of which generate graphs according to different probability distributions. The first model is parameterized by a given edge probability p. Typically, p=0.5, although smaller values can be used to construct sparser random graphs. In this model a coin is flipped for each pair of vertices x and y to decide whether to add an edge (x,y). All labeled graphs will be generated with equal probability when p=1/2.û™Z ?¯?b ’€7€Ȁ€€€€€€€€†"€€€ç `4"€†"€‰€‚ÿThe second model is parameterized by the desired number of edges m. It selects m distinct edges uniformly at random. One way to do this is by drawing random (x,y)-pairs and creating an edge if that pair is not already in the graph.   An alternative approach to computing the same things constructs the set of possible edges and selects a random m-subset of them, as discussed in Section .õ¶Ž ?°D?? L€m €€‚€€€€€€€€‚ÿWhich of these o¯?°D? ?ptions best models your application? Probably none of them. Random graphs, by definition, have very little structure. In most applications, graphs are used to model relationships, which are often highly structured. Experiments conducted on random graphs, although interesting and easy to perform, often fail to capture what you are looking for.An alternative to random graphs is to use ``organic'' graphs, graphs that reflect the relationships among real-world objects.     The Stanford GraphBase, discussed below, is an outstanding source of organic graphs. Further, there are many raw sources of relationships electronically available via the Internet that can be turned into interesting organic graphs with a little programming and imagination.   Consider the graph defined by a set of WWW pages, with any hyperlink between two pages defining an edge.   Or what about the graph implicit in railroad, subway, or airline networks, with vertices being stations and edges between two stations connected by direct service?   As a final example, every large computer program defines a call graph, where the vertices represent subroutines, and there is an edge (x,y) if x calls y.’n¯?BE?$ €Ü€€‚ÿTwo special classes of graphs have generation algorithms that have proven particularly useful in practice:ˆ °D?ÊF?h ž€G€Ȁ€€€€€ç©Ñ-À†"€‰€†"€€€€€†"€‚ÿu Trees -     Prüfer codes provide a simple way to rank and unrank labeled trees and thus solve all the standard generation problems discussed in Section .   There are exactly labeled trees on n vertices, and exactly that many strings of length n-2 on the alphabet .RðBE?J?b ’€å€Ȁ€€€€†"€Ȁ€€€€€€€€€†"€ȂÿThe key to Prüfer's bijection is the observation that every tree has at least two vertices of degree 1.   Thus in any labeled tree, the vertex v incident on the leaf with lowest label is well-defined. We take v to be , the first character in the code. We then delete the associated leaf and repeat the procedure until only two vertices are left. This defines a unique code S for any given labeled tree that can be used to rank the tree. To go from code to tree, observe that the degree of vertex v in the tree is one more than the number of times v occurs in S. The lowest-labeled leaf will be the smallest integer missing from S, which when paired with determines the first edge of the tree. The entire tree follows by induction.žÊF?,L?r ²€A€Ȁ‚€€€€€€€†"€–†"€&€€€€€€€€€€€€‚ÿAlgorithms for efficiently generating unlabeled rooted trees are presented in the implementation section below.u Fixed degree sequence graphs -     The degree sequence of a graph G is an integer partition where is the degree of the ith highest-degree vertex of G.    Since each edge contributes to the degree of two vertices, p is a partition of 2m, where m is the number of edges in G.ô›J? O?Y €€A€Ȁ†"€m†"€#€€†"€†"€#†"€‚ÿNot all partitions correspond to degree sequences of graphs. However, there is a recursive construction that constructs a graph with a given degree sequence if one exists. If a partition is realizable, the highest-degree vertex can be connected to the next highest-degree vertices in G, or the vertices corresponding to parts . Deleting and decrementing yields a smaller partition, which we recur on. If we terminate without ever creating negative numbers, the partition was realizable. Since we always connect the highest-degree vertex to other high-degree vertices, it is important to reorder the parts of the partition by size after each iteration.ðˆ,L??h ž€€Ȁ€€€€€€€€€€€€€€€€€€€€€€‚ÿAlthough this construction is deterministic, a semirandom collection of graphs realizing this degree sequence   can be O?? ? generated from G using edge-flipping operations. Suppose edges (x,y) and (w,z) are in G, but (x,w) and (y,z) are not. Exchanging these pairs of edges creates a different (not necessarily connected) graph without changing the degrees of any vertex.V O?r„?R r€ €€æ”ÛÛ/€‰€璑K€‰€ç9.4^€†"€‰€‚ÿImplementations: The Stanford GraphBase [Knu94]   is perhaps most useful as an instance generator for constructing a wide variety of graphs to serve as test data for other programs. It incorporates graphs derived from interactions of characters in famous novels, Roget's Thesaurus, the Mona Lisa, expander graphs, and the economy of the United States. It also contains routines for generating binary trees, graph products, line graphs, and other operations on basic graphs.   Finally, because of its machine-independent random number generators, it provides a way to construct random graphs such that they can be reconstructed elsewhere, thus making them perfect for experimental comparisons of algorithms. See Section for additional information.        Ç?‹†?R r€‘€€ç˜K€‰€碂K€‰€皌:œ€†"€‰€‚ÿCombinatorica [Ski90] provides Mathematica generators for basic graphs such as stars, wheels, complete graphs, random graphs and trees, and graphs with a given degree sequence.     Further, it includes operations to construct more interesting graphs from these, including join, product, and line graph.   Graffiti [Faj87], a collection of almost 200 graphs of graph-theoretic interest, are available in Combinatorica format. See Section .Œr„?Œ‰?u ž€€€çó‘¶ü€†"€‰€‚ç?‘K€‰€çÊnuŽ€†"€‰€çƒ A€†"€‰€€‚ÿThe graph isomorphism testing program nauty (see Section ),     by Brendan D. McKay of the Australian National University, has been used to generate catalogs of all nonisomorphic graphs with up to 11 vertices.   This extension to nauty, named makeg, can be obtained by anonymous ftp from bellatrix.anu.edu.au (150.203.23.14) in the directory pub/nauty19.Nijenhuis and Wilf [NW78] provide efficient Fortran routines to enumerate all labeled trees via Prüfer codes and to construct random unlabeled rooted trees. See Section .     A collection of generators for standard families of graphs is included with LEDA (see Section ). Ž(‹†?Œ?f š€Q€€€ç‰K€‰€矘K€‰€煉K€‰€‚çy‚K€‰€çj‘K€‰€‚ÿNotes: An extensive literature exists on generating graphs uniformly at random. Surveys include [Gol93, Tin90]. Closely related to the problem of generating classes of graphs is counting them.   Harary and Palmer [HP73] survey results in graphical enumeration.Random graph theory is concerned with the properties of random graphs. Threshold laws in random graph theory define the edge density at which properties such as connectedness become highly likely to occur.   Expositions on random graph theory include [ES74, Pal85].¢Œ‰?9Ž?} ȀG€€€€çN‚K€‰€€€€€‚†"€‚€€çG—K€‰€矂K€‰€ç?‘K€‰€€‚ÿAn integer partition is graphic if there exists a simple graph with that degree sequence.     Erdős and Gallai [EG60] proved that a degree sequence is graphic if and only if the sequence observes the following condition for each integer r < n:The bijection between n-2 strings and labeled trees is due to Prüfer [Prü18].   Good expositions on this result include [Eve79a, NW78]. œiŒ?öŽ?T x€Ö€€€ç©Ñ-À†"€‰€çó‘¶ü€†"€‰€€‚ÿRelated Problems: Generating permutations (see page ), graph isomorphism (see page ).    )9Ž??& €€€‚ÿÈÇöŽ?óÀ? ЁŸ€€‚çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æ•ÛÛ/€†"€‰€æ–ÛÛ/€†"€‰€æ—ÛÛ/€†"€‰€€çg_ɀ‰€€çÚf?óÀ? ?_ɀ‰€€çg_ɀ‰€€‚‚ÿ Next: Calendrical Calculations Up: Combinatorial Problems Previous: Generating Partitions AlgorithmsMon Jun 2 23:33:50 EDT 1997)?Á?& €€€‚ÿµfóÀ?ÑÁ?O n€Ì€€Ì#EF("http://www.webtwin.com/",`',1)€ ‰€ÿThis help file was generated from HTML source using an unregistered copy of WebTwin version 1.1b1Á?Â?1Uÿÿÿÿÿÿÿÿ/ÿÿÿÿÂ?&Â?$ÑÁ?&Â?" €€€ ÿIÂ?oÂ?1€ÿÿÿÿÿÿÿÿ0oÂ?®Â?â @Calendrical Calculations?&Â?®Â?# €8€€‚ÿCalendrical Calculations 1oÂ?ßÄ?! ‚3€€çg_ɀ†"€ ‰€çÚf_ɀ†"€ ‰€çg_ɀ†"€ ‰€çaxÊ«€†"€ ‰€çbxÊ«€†"€ ‰€æÃÛÛ/€†"€‰€æºÛÛ/€†"€‰€æ»ÛÛ/€†"€‰€€çg_ɀ‰€€çÚf_ɀ‰€€çg_ɀ‰€‚€ ‚€‚†"€€‚€€€€‚ÿ Next: Job Scheduling Up: Combinatorial Problems Previous: Generating GraphsCalendrical Calculations    Input description: A particular calendar date d, specified by month, day, and year. —a®Â?vÇ?6 :€Ã€€€€€€‚€‚ÿProblem description: Which day of the week did d fall on according to the given calendar system? Discussion: Many business applications need to perform calendrical calculations. Perhaps we want to display a calendar of a specified month and year. Maybe we need to compute what day of the week or year some event occurs, as in figuring out the date on which a 180-day futures contract comes due. The importance of correct calendrical calculations is perhaps best revealed by the furor over the ``millennium bug,'' the crisis in legacy programs that allocate only two digits for storing the year.    êßÄ?‡Ê?' €Õ€€‚‚ÿMore complicated questions arise in international applications, because different nations and ethnic groups around the world use different calendar systems. Some of these, like the Gregorian calendar used in most of the world, are based on the sun, while others, like the Hebrew calendar, are lunar calendars. How would you tell today's date according to the Chinese or Arabic calendar?    The algorithms associated with calendrical calculations are different from the other problems in this book, because calendars are historical objects, not mathematical ones. The issues revolve around specifying the rules of th