1 00:00:00,500 --> 00:00:03,130 All right, so what we're going to do today 2 00:00:03,130 --> 00:00:05,630 is continue our discussion of modularity 3 00:00:05,630 --> 00:00:08,970 and how you hook software modules up together. 4 00:00:08,970 --> 00:00:11,110 And what we did the last time was 5 00:00:11,110 --> 00:00:14,760 talked about how you share data between programs between users. 6 00:00:14,760 --> 00:00:17,800 And we went to the design of the UNIX file system, 7 00:00:17,800 --> 00:00:20,580 or at least a particular aspect of the UNIX file system 8 00:00:20,580 --> 00:00:23,940 where we talked about how we built a file system out 9 00:00:23,940 --> 00:00:25,200 of layers and layers. 10 00:00:25,200 --> 00:00:29,440 And each layer was essentially doing name resolution in order 11 00:00:29,440 --> 00:00:33,040 for us to take a UNIX path name and eventually 12 00:00:33,040 --> 00:00:35,060 get the data blocks corresponding 13 00:00:35,060 --> 00:00:38,020 to the data in that file. 14 00:00:38,020 --> 00:00:41,759 What we're going to do today is to move away a little bit from 15 00:00:41,759 --> 00:00:44,050 the memory abstraction, which is what we spend our time 16 00:00:44,050 --> 00:00:47,530 on the last time, and start talking about the second 17 00:00:47,530 --> 00:00:49,860 abstraction -- the interpreter abstraction. 18 00:00:49,860 --> 00:00:52,360 And we're going to do that in the context of software 19 00:00:52,360 --> 00:00:55,220 libraries, and basically understand 20 00:00:55,220 --> 00:00:58,500 how software libraries are put together, 21 00:00:58,500 --> 00:01:00,970 and how you can build large software programs out 22 00:01:00,970 --> 00:01:05,150 of individual software modules that get hooked together. 23 00:01:05,150 --> 00:01:07,800 So that's the plan for today. 24 00:01:07,800 --> 00:01:10,460 And in addition to seeing the actual mechanism for how 25 00:01:10,460 --> 00:01:13,300 you take all these modules in the form of libraries 26 00:01:13,300 --> 00:01:15,770 and other software pieces, hook them up together, 27 00:01:15,770 --> 00:01:17,270 what we are really going to focus on 28 00:01:17,270 --> 00:01:21,410 is the mechanism by which these different modules actually 29 00:01:21,410 --> 00:01:23,250 run on your computer. 30 00:01:23,250 --> 00:01:26,149 OK, so it's going to involve a fair amount of mechanism going 31 00:01:26,149 --> 00:01:27,940 down to the lowest layer until you actually 32 00:01:27,940 --> 00:01:30,590 have something that's a single, executable file that 33 00:01:30,590 --> 00:01:32,720 runs on your computer. 34 00:01:32,720 --> 00:01:35,380 And then what we're going to do is 35 00:01:35,380 --> 00:01:39,120 to step back and think about the kind of modularity 36 00:01:39,120 --> 00:01:42,380 that we will have gotten from this approach. 37 00:01:42,380 --> 00:01:44,190 And that's the kind of modularity 38 00:01:44,190 --> 00:01:46,930 that we're going to call soft modularity. 39 00:01:49,770 --> 00:01:51,180 So, that's the plan for today. 40 00:01:51,180 --> 00:01:53,970 And you'll see why this is called soft modularity. 41 00:01:53,970 --> 00:01:58,820 And it has to do with the way in which propagation of faults 42 00:01:58,820 --> 00:02:00,906 occurs and in particular the kind of modularity 43 00:02:00,906 --> 00:02:03,030 at the libraries that we are going to discuss today 44 00:02:03,030 --> 00:02:07,360 has the property that a problem in one module, for example, 45 00:02:07,360 --> 00:02:09,801 an infinite loop or a crash in one module 46 00:02:09,801 --> 00:02:11,550 will turn out to affect the whole program. 47 00:02:11,550 --> 00:02:13,500 And the whole thing will come crumbling down. 48 00:02:13,500 --> 00:02:16,180 OK, and that's why it's going to be called soft modularity. 49 00:02:16,180 --> 00:02:18,400 Of course, we don't like that kind of modularity, 50 00:02:18,400 --> 00:02:20,440 although it is useful to have soft modularity. 51 00:02:20,440 --> 00:02:23,960 So the next three lectures after today, we're 52 00:02:23,960 --> 00:02:26,690 going to talk about hardening the soft modularity using 53 00:02:26,690 --> 00:02:28,310 a variety of different techniques. 54 00:02:28,310 --> 00:02:33,020 So that's the plan for the next three to four lectures. 55 00:02:33,020 --> 00:02:36,770 OK, so let's take some examples to start with of where you end 56 00:02:36,770 --> 00:02:39,090 up using these modules -- these software modules -- 57 00:02:39,090 --> 00:02:41,077 to build bigger software systems. 58 00:02:41,077 --> 00:02:42,660 And the first one is actually what you 59 00:02:42,660 --> 00:02:44,230 see in programming language. 60 00:02:44,230 --> 00:02:50,980 When you use C or C++ or Java or Perl or any of these programs 61 00:02:50,980 --> 00:02:54,030 that you're familiar with, you end up actually building large 62 00:02:54,030 --> 00:02:56,590 programs out of taking lots of little programs, 63 00:02:56,590 --> 00:02:59,060 usually programmed in different files, 64 00:02:59,060 --> 00:03:03,350 and running a compiler on it, and it'll turn out to require 65 00:03:03,350 --> 00:03:06,500 other system software to take programs that are written 66 00:03:06,500 --> 00:03:08,620 in different modules and hook them all up together 67 00:03:08,620 --> 00:03:10,606 into a bigger program. 68 00:03:10,606 --> 00:03:11,980 And this is actually going to be, 69 00:03:11,980 --> 00:03:15,480 we're going to learn today with examples from C or C++ 70 00:03:15,480 --> 00:03:18,650 programming language to find out how we build C modules 71 00:03:18,650 --> 00:03:19,410 together. 72 00:03:19,410 --> 00:03:21,110 But, by no means are we restricted 73 00:03:21,110 --> 00:03:23,220 to these programming languages. 74 00:03:23,220 --> 00:03:26,480 In fact, there's a lot of different examples. 75 00:03:26,480 --> 00:03:29,415 Database systems provide a great example of modularity. 76 00:03:32,690 --> 00:03:34,560 So if you take a system like, you 77 00:03:34,560 --> 00:03:36,150 might've heard of Oracle or Sybase, 78 00:03:36,150 --> 00:03:38,940 or any of IBM's DB2, all of these things 79 00:03:38,940 --> 00:03:42,200 are fairly complicated pieces of software. 80 00:03:42,200 --> 00:03:43,930 But what they invariably allow you to do 81 00:03:43,930 --> 00:03:48,109 is to put in your own software, your own code, that will run 82 00:03:48,109 --> 00:03:49,400 as part of the database system. 83 00:03:49,400 --> 00:03:51,710 So if you, for example, come up with, 84 00:03:51,710 --> 00:03:53,970 you know, normally databases allow you to run queries 85 00:03:53,970 --> 00:03:55,245 on tables of data. 86 00:03:58,770 --> 00:04:00,620 We'll be back after these messages. 87 00:04:00,620 --> 00:04:03,580 OK, so every database system allows you to upload modules 88 00:04:03,580 --> 00:04:05,690 into it that run as part of the database system. 89 00:04:05,690 --> 00:04:09,090 So if you come up with a different kind of data type, 90 00:04:09,090 --> 00:04:13,020 for example, some geometric data type on which you asked 91 00:04:13,020 --> 00:04:16,709 whether there is a bunch of points inside a polygon, 92 00:04:16,709 --> 00:04:19,339 a regular database system won't actually support that. 93 00:04:19,339 --> 00:04:21,550 But what you can do is write code for that, 94 00:04:21,550 --> 00:04:25,810 and upload it, and run it as a module in a database system. 95 00:04:25,810 --> 00:04:28,760 Another example, a common example these days are Web 96 00:04:28,760 --> 00:04:33,360 servers, particularly something like Apache -- 97 00:04:33,360 --> 00:04:38,800 -- where you can actually include as part of Apache when 98 00:04:38,800 --> 00:04:41,550 you run it a bunch of different modules for doing a variety 99 00:04:41,550 --> 00:04:44,780 of new functions that weren't previously running. 100 00:04:44,780 --> 00:04:47,350 And all of these different kinds of systems 101 00:04:47,350 --> 00:04:49,780 will turn out to use essentially variants 102 00:04:49,780 --> 00:04:53,580 of the same basic idea in terms of solving 103 00:04:53,580 --> 00:04:56,960 the problem of taking these different modules together 104 00:04:56,960 --> 00:05:01,600 and composing a single, large program that you can execute 105 00:05:01,600 --> 00:05:04,930 on your computer. 106 00:05:04,930 --> 00:05:11,050 So what we're going to do today is discuss this method, 107 00:05:11,050 --> 00:05:13,830 this way of doing modularity in the context of, 108 00:05:13,830 --> 00:05:16,190 a particular example, it's easiest to do 109 00:05:16,190 --> 00:05:17,456 this with an example. 110 00:05:17,456 --> 00:05:18,830 And then we'll step back and talk 111 00:05:18,830 --> 00:05:20,130 about the general principles. 112 00:05:20,130 --> 00:05:25,550 So, we're going to focus on the Linux operating 113 00:05:25,550 --> 00:05:27,084 system running the GNU tools. 114 00:05:27,084 --> 00:05:29,500 And in particular, we're going to focus on how you do this 115 00:05:29,500 --> 00:05:32,210 in a language like C or C++. 116 00:05:32,210 --> 00:05:35,750 And so, most of you are probably familiar with some of the tools 117 00:05:35,750 --> 00:05:39,030 that you use to build programs in GNU and Linux 118 00:05:39,030 --> 00:05:41,630 and we are going to use a compiler called 119 00:05:41,630 --> 00:05:45,290 GCC that will turn out to use some other pieces of software 120 00:05:45,290 --> 00:05:47,460 to build modular programs. 121 00:05:47,460 --> 00:05:50,967 So, that's kind of what we're going to start with. 122 00:05:50,967 --> 00:05:53,550 And the basic approach in terms of how these modules are going 123 00:05:53,550 --> 00:05:57,300 to be written is if you want to write a program, you typically 124 00:05:57,300 --> 00:06:00,060 design what you want to do, design the system, 125 00:06:00,060 --> 00:06:03,820 and then decide on how you decompose 126 00:06:03,820 --> 00:06:06,160 the functionality of your system into a bunch 127 00:06:06,160 --> 00:06:07,200 of different modules. 128 00:06:07,200 --> 00:06:10,675 And typically you write one or more files for each module 129 00:06:10,675 --> 00:06:13,300 and system, and then you need a way to bring all these modules, 130 00:06:13,300 --> 00:06:16,240 these different files together to build a bigger program. 131 00:06:16,240 --> 00:06:20,760 And when you run GCC in particular on a dot O file, 132 00:06:20,760 --> 00:06:23,350 the modules themselves want to compile a C file. 133 00:06:23,350 --> 00:06:25,730 As you know, you produce an object file 134 00:06:25,730 --> 00:06:32,770 with a name like file dot O, and that actually contains in it, 135 00:06:32,770 --> 00:06:35,700 that's an object file that with some more work 136 00:06:35,700 --> 00:06:39,464 you can arrange to run on your computer. 137 00:06:39,464 --> 00:06:41,380 And what we're going to do today is figure out 138 00:06:41,380 --> 00:06:45,640 how when you have a large number of object files, 139 00:06:45,640 --> 00:06:48,310 and it will also turn out, we'll talk about something called 140 00:06:48,310 --> 00:06:51,000 a library, which is a collection of object files put 141 00:06:51,000 --> 00:06:54,440 into a single archive, or how you can take those object files 142 00:06:54,440 --> 00:06:57,950 and produce an executable that will run. 143 00:06:57,950 --> 00:07:00,460 So if this works, let me show you 144 00:07:00,460 --> 00:07:02,910 an example of what we're going to be talking about. 145 00:07:02,910 --> 00:07:10,507 So I have two little files, a little trivia program. 146 00:07:10,507 --> 00:07:11,840 I don't know if this is visible. 147 00:07:15,220 --> 00:07:16,810 OK, so all this is doing is computing 148 00:07:16,810 --> 00:07:18,630 the square of the number. 149 00:07:18,630 --> 00:07:23,790 And it uses here, this is a very trivial program. 150 00:07:23,790 --> 00:07:26,820 It defines a few variables, and then just 151 00:07:26,820 --> 00:07:30,280 calls on modular function called SQR, which isn't actually 152 00:07:30,280 --> 00:07:30,890 in this file. 153 00:07:30,890 --> 00:07:32,590 It's in a separate file. 154 00:07:32,590 --> 00:07:35,550 But one thing you can do is run GCC-C 155 00:07:35,550 --> 00:07:38,040 which just tells it to produce the object file, 156 00:07:38,040 --> 00:07:42,817 and there's a program called object dump which 157 00:07:42,817 --> 00:07:44,400 is a useful program if you want to see 158 00:07:44,400 --> 00:07:45,786 what's inside your object file. 159 00:07:45,786 --> 00:07:47,160 It's a binary file, so you're not 160 00:07:47,160 --> 00:07:48,950 going to be able to hack it. 161 00:07:48,950 --> 00:07:57,630 But you could run it with, we've already reached the bottom, 162 00:07:57,630 --> 00:07:58,170 all right. 163 00:08:01,110 --> 00:08:04,870 OK, so the part to worry about for the moment 164 00:08:04,870 --> 00:08:07,120 are shown in the last three lines 165 00:08:07,120 --> 00:08:12,090 that you could see here starting main, SQR, and printf. 166 00:08:12,090 --> 00:08:14,270 And those are the three functions 167 00:08:14,270 --> 00:08:16,990 that are being invoked by the program I showed you. 168 00:08:16,990 --> 00:08:19,210 And you can see, for SQR and printf, 169 00:08:19,210 --> 00:08:22,506 this object file doesn't actually know where it is 170 00:08:22,506 --> 00:08:23,630 or where its definition is. 171 00:08:23,630 --> 00:08:26,830 And that's why you see UND for undefined. 172 00:08:26,830 --> 00:08:29,460 And part of what we're going to figure out today 173 00:08:29,460 --> 00:08:33,070 is how we can find out where those other modules are 174 00:08:33,070 --> 00:08:36,250 defined, and hook up all those different modules together 175 00:08:36,250 --> 00:08:38,049 to build a program that will actually run 176 00:08:38,049 --> 00:08:39,860 to do what we want it to do. 177 00:08:42,720 --> 00:08:45,330 So the other thing here is square dot C, 178 00:08:45,330 --> 00:08:48,170 which does the obvious thing of just multiplying two numbers. 179 00:08:48,170 --> 00:08:53,970 And you could do the same thing with square dot C. 180 00:08:53,970 --> 00:08:56,520 So I have a little thing called show object which is just 181 00:08:56,520 --> 00:08:57,660 an alias for object dump. 182 00:09:02,700 --> 00:09:04,870 And you can see here that there aren't actually 183 00:09:04,870 --> 00:09:08,120 any undefined symbols or undefined functions 184 00:09:08,120 --> 00:09:10,210 because square, in turn, didn't invoke 185 00:09:10,210 --> 00:09:14,790 anything that was outside and defined somewhere else. 186 00:09:14,790 --> 00:09:18,500 And underneath here, you see the actual disassembled assembler 187 00:09:18,500 --> 00:09:23,230 code for this machine text. 188 00:09:23,230 --> 00:09:27,670 So going back to M dot O, I just want 189 00:09:27,670 --> 00:09:36,160 to draw your attention to a couple of things. 190 00:09:36,160 --> 00:09:36,660 Yeah? 191 00:09:36,660 --> 00:09:40,980 Shows up fine on my screen, I know. 192 00:09:40,980 --> 00:09:43,890 [LAUGHTER] All right, good. 193 00:09:47,700 --> 00:09:51,650 OK, so there's a few sections to this object file here. 194 00:09:51,650 --> 00:09:54,240 And it's important to understand a little bit 195 00:09:54,240 --> 00:09:56,360 how an object file is laid out. 196 00:09:56,360 --> 00:09:59,410 It has in it a few different sections as I mentioned. 197 00:09:59,410 --> 00:10:02,220 One of the sections is called the text section. 198 00:10:02,220 --> 00:10:06,050 The text section basically contains the machine code. 199 00:10:06,050 --> 00:10:09,310 I mean, ideally once you hook all these modules together, 200 00:10:09,310 --> 00:10:14,209 that machine code will actually just run on your computer. 201 00:10:14,209 --> 00:10:16,500 It also has a section, I think you can see it up there, 202 00:10:16,500 --> 00:10:18,280 called auto data. 203 00:10:18,280 --> 00:10:23,000 That's the second section that stands for read only data. 204 00:10:23,000 --> 00:10:25,340 And one of the things I had here in that program 205 00:10:25,340 --> 00:10:27,902 was something that said printf with a string inside. 206 00:10:27,902 --> 00:10:29,360 And, that string is read only data. 207 00:10:29,360 --> 00:10:31,690 I mean, you don't want, the program doesn't actually 208 00:10:31,690 --> 00:10:32,450 modify it. 209 00:10:32,450 --> 00:10:34,060 And that's the kind of thing you could see it on the right. 210 00:10:34,060 --> 00:10:35,580 There is a comment there: the square of something 211 00:10:35,580 --> 00:10:36,350 is something. 212 00:10:36,350 --> 00:10:38,740 And that's an example of read only data. 213 00:10:38,740 --> 00:10:41,250 And then it has a section for data 214 00:10:41,250 --> 00:10:43,810 which really corresponds to the global variables used 215 00:10:43,810 --> 00:10:46,280 in the program. 216 00:10:46,280 --> 00:10:48,480 And for those of you who remember 6.004, 217 00:10:48,480 --> 00:10:51,159 and if not we'll talk about this next time. 218 00:10:51,159 --> 00:10:52,700 The look of variables in your modules 219 00:10:52,700 --> 00:10:54,241 are not actually in the data section. 220 00:10:54,241 --> 00:10:55,770 They're typically on the stack. 221 00:10:55,770 --> 00:10:59,820 So we're not going to worry about that for today. 222 00:10:59,820 --> 00:11:02,470 And then there's a section called a symbol 223 00:11:02,470 --> 00:11:06,590 table which is shown here, I think, as sym tab 224 00:11:06,590 --> 00:11:07,585 not on the screen. 225 00:11:07,585 --> 00:11:08,210 Let me go back. 226 00:11:11,280 --> 00:11:12,790 So that's the symbol table. 227 00:11:12,790 --> 00:11:18,330 And what this symbol table shows is 228 00:11:18,330 --> 00:11:20,700 the different global variables and functions 229 00:11:20,700 --> 00:11:24,360 that are either defined in the module in this dot O file 230 00:11:24,360 --> 00:11:27,270 or are referenced by this module. 231 00:11:27,270 --> 00:11:30,910 OK, and for the symbols that are defined in this module, what 232 00:11:30,910 --> 00:11:33,990 the symbol table tells you is the address at which you 233 00:11:33,990 --> 00:11:35,780 can find it in the module. 234 00:11:35,780 --> 00:11:38,850 OK, and for the things that are not defined, 235 00:11:38,850 --> 00:11:40,040 it says it's undefined. 236 00:11:40,040 --> 00:11:43,760 And hopefully once the compiler sees 237 00:11:43,760 --> 00:11:45,850 all of the different object files involved, 238 00:11:45,850 --> 00:11:48,060 it can piece together these different object files 239 00:11:48,060 --> 00:11:51,322 and build a larger program module. 240 00:11:51,322 --> 00:11:53,780 So when you compile each of these things in this example, M 241 00:11:53,780 --> 00:11:56,270 dot O and SQR dot O, you will find 242 00:11:56,270 --> 00:11:59,480 that the object file that's produced for both files 243 00:11:59,480 --> 00:12:01,846 starts with address zero. 244 00:12:01,846 --> 00:12:04,220 OK, so obviously when you hook these modules up together, 245 00:12:04,220 --> 00:12:07,090 you can't have both modules run at address zero. 246 00:12:07,090 --> 00:12:09,040 So one of the things you need to be able to do 247 00:12:09,040 --> 00:12:11,790 is to take these different object files together, 248 00:12:11,790 --> 00:12:16,110 and somehow join them so that the addresses are no longer 249 00:12:16,110 --> 00:12:18,270 colliding with one another. 250 00:12:18,270 --> 00:12:22,420 So that's one problem that we are going to have to solve. 251 00:12:22,420 --> 00:12:26,410 So that's what we're going to do today. 252 00:12:26,410 --> 00:12:30,700 So there's basically three steps that we 253 00:12:30,700 --> 00:12:32,810 are going to talk about today. 254 00:12:32,810 --> 00:12:36,590 The first step is figuring out all these different symbols. 255 00:12:36,590 --> 00:12:39,480 So when you have SQR or if you were using square root 256 00:12:39,480 --> 00:12:42,010 in a different program, printf is another example, 257 00:12:42,010 --> 00:12:44,240 figuring out where the definitions of these symbols 258 00:12:44,240 --> 00:12:47,410 are, and where the definitions of the global variables 259 00:12:47,410 --> 00:12:48,499 are in your program. 260 00:12:48,499 --> 00:12:50,290 And that's a step called symbol resolution. 261 00:12:55,810 --> 00:12:59,120 OK, and the plan is going to be that each object file is going 262 00:12:59,120 --> 00:13:02,000 to have inside it a table like here 263 00:13:02,000 --> 00:13:07,390 that shows for the symbols that have been locally defined, 264 00:13:07,390 --> 00:13:09,240 where in the local module they are. 265 00:13:09,240 --> 00:13:12,200 I'm going to use the word module for these dot O files. 266 00:13:12,200 --> 00:13:15,290 And for symbols that are not in the same dot O file, 267 00:13:15,290 --> 00:13:16,720 it just says it's undefined. 268 00:13:16,720 --> 00:13:17,470 And I need it. 269 00:13:17,470 --> 00:13:20,280 OK, and that's what the dot O contains. 270 00:13:20,280 --> 00:13:24,210 And, when you take all these dot O's together and produce 271 00:13:24,210 --> 00:13:29,030 a bigger program, those symbols are going to be resolved. 272 00:13:29,030 --> 00:13:32,840 And that's the step called symbol resolution. 273 00:13:32,840 --> 00:13:36,090 The second thing we have to do before we 274 00:13:36,090 --> 00:13:38,805 get a single big program is to do something called relocation. 275 00:13:43,960 --> 00:13:45,660 So what relocation is, is remember 276 00:13:45,660 --> 00:13:48,540 I told you when you compile a program to a dot O file, 277 00:13:48,540 --> 00:13:52,436 all of the dot O files all have addresses starting from zero. 278 00:13:52,436 --> 00:13:54,560 You can't run them all together unless you actually 279 00:13:54,560 --> 00:13:56,024 modify the different addresses. 280 00:13:56,024 --> 00:13:58,690 And any time you see a reference to a variable in one of the dot 281 00:13:58,690 --> 00:14:01,560 O files, you have to modify that address 282 00:14:01,560 --> 00:14:05,440 to point to the actual address at which the instruction would 283 00:14:05,440 --> 00:14:07,610 run or the data object is present. 284 00:14:07,610 --> 00:14:09,550 And that's called relocation. 285 00:14:09,550 --> 00:14:12,172 And it'll turn out that the object files 286 00:14:12,172 --> 00:14:14,380 that are produced by the compiler are what are called 287 00:14:14,380 --> 00:14:15,600 relocatable object files. 288 00:14:15,600 --> 00:14:17,933 Pretty much every object file these days is relocatable. 289 00:14:17,933 --> 00:14:19,680 But the relocatable object file allows 290 00:14:19,680 --> 00:14:22,340 you to do this relocation easily because it 291 00:14:22,340 --> 00:14:26,700 has in it information that tells whoever is composing 292 00:14:26,700 --> 00:14:29,590 these modules together how to modify the addresses that are 293 00:14:29,590 --> 00:14:30,880 referenced within each module. 294 00:14:34,080 --> 00:14:36,690 And the third step once you do all this 295 00:14:36,690 --> 00:14:38,490 and you produce a single big program that's 296 00:14:38,490 --> 00:14:41,090 ready to execute, is actually to run the program. 297 00:14:41,090 --> 00:14:47,190 And that's a step called loading or program loading. 298 00:14:47,190 --> 00:14:49,330 So you create a big executable file, 299 00:14:49,330 --> 00:14:51,860 and you type it on the command line. 300 00:14:51,860 --> 00:14:54,160 What happens? 301 00:14:54,160 --> 00:14:56,230 So, those are the three steps that we are going 302 00:14:56,230 --> 00:14:57,950 to be talking about today. 303 00:14:57,950 --> 00:15:01,800 And the first two steps typically are called linking. 304 00:15:01,800 --> 00:15:08,350 OK, so most of today is going to be talking 305 00:15:08,350 --> 00:15:09,510 about how linking works. 306 00:15:09,510 --> 00:15:11,301 And the piece of software, system software, 307 00:15:11,301 --> 00:15:13,230 that does linking is called a linker. 308 00:15:13,230 --> 00:15:16,150 And in the GNU-Linux operating system, 309 00:15:16,150 --> 00:15:19,010 it's a program called LD. 310 00:15:19,010 --> 00:15:20,400 That's the linking program. 311 00:15:20,400 --> 00:15:23,590 And so when you run GCC to take a bunch of modules 312 00:15:23,590 --> 00:15:27,310 and produce a program out of it, underneath inside 313 00:15:27,310 --> 00:15:29,960 not visible to you but underneath, GCC actually 314 00:15:29,960 --> 00:15:31,760 invokes a bunch of other programs including 315 00:15:31,760 --> 00:15:34,350 LD as one of the last steps to produce 316 00:15:34,350 --> 00:15:35,940 an actual file that will run. 317 00:15:38,910 --> 00:15:42,360 So the linker takes as input, object files, 318 00:15:42,360 --> 00:15:46,640 and produces as output a program that you can run. 319 00:15:46,640 --> 00:15:51,750 And then the loader takes over and runs the program. 320 00:15:51,750 --> 00:15:54,900 So there are actually three kinds of object files 321 00:15:54,900 --> 00:15:58,170 that the linker takes as argument. 322 00:15:58,170 --> 00:16:01,870 The first kind of object file is a relocatable object file. 323 00:16:08,040 --> 00:16:10,000 OK, actually there's two kinds. 324 00:16:10,000 --> 00:16:11,882 The first kind is a relocatable object file. 325 00:16:11,882 --> 00:16:14,340 And that's what we're going to be spending most of our time 326 00:16:14,340 --> 00:16:14,970 on. 327 00:16:14,970 --> 00:16:16,826 The second kind of object file is something 328 00:16:16,826 --> 00:16:17,950 called a share object file. 329 00:16:22,740 --> 00:16:24,951 We are going to get to this a little bit at the end. 330 00:16:24,951 --> 00:16:26,950 And the reason we're going to wait until the end 331 00:16:26,950 --> 00:16:29,320 is that the reason for a shared object file 332 00:16:29,320 --> 00:16:33,390 is that if you have a program, a module like printf or some math 333 00:16:33,390 --> 00:16:36,860 library like using functions like square root, 334 00:16:36,860 --> 00:16:40,910 if you just do linking the normal way, which 335 00:16:40,910 --> 00:16:44,350 is the easy way we're going to talk about for most of today, 336 00:16:44,350 --> 00:16:48,950 you will end up with a program that includes in its text that 337 00:16:48,950 --> 00:16:52,340 includes in the binary all of the modules 338 00:16:52,340 --> 00:16:54,330 corresponding to all of the libraries at, 339 00:16:54,330 --> 00:16:57,189 you know, the text corresponds to the different modules 340 00:16:57,189 --> 00:16:57,730 that you use. 341 00:16:57,730 --> 00:16:59,450 So now, if you have 100 different programs running 342 00:16:59,450 --> 00:17:01,330 on your system, all of which use printf, 343 00:17:01,330 --> 00:17:04,380 it will turn out that every program is quite big. 344 00:17:04,380 --> 00:17:08,720 And so, shared object files allow us to not actually have 345 00:17:08,720 --> 00:17:12,380 to include the text of printf in every binary that we produce, 346 00:17:12,380 --> 00:17:14,839 but just maintain a pointer to something that says, 347 00:17:14,839 --> 00:17:17,589 when you need printf, go and get this, and include it 348 00:17:17,589 --> 00:17:19,050 as part of your program. 349 00:17:19,050 --> 00:17:20,500 So we get to that in the end. 350 00:17:20,500 --> 00:17:21,958 But for the most part, we are going 351 00:17:21,958 --> 00:17:24,710 to be focusing on relocatable object files, that is, 352 00:17:24,710 --> 00:17:27,310 object files that are produced by something like GCC 353 00:17:27,310 --> 00:17:33,430 that I showed you, which include in it information for modifying 354 00:17:33,430 --> 00:17:36,620 the addresses that are being referenced inside the module. 355 00:17:36,620 --> 00:17:39,150 And once the linker takes these as arguments, 356 00:17:39,150 --> 00:17:44,100 what it produces as output, out comes an executable file. 357 00:17:48,230 --> 00:17:49,960 And these dot O's don't actually run. 358 00:17:49,960 --> 00:17:52,910 The executable is a thing that's capable of actually running. 359 00:18:01,610 --> 00:18:03,420 OK, so the first step we have to talk about 360 00:18:03,420 --> 00:18:04,295 is symbol resolution. 361 00:18:12,570 --> 00:18:14,396 And so the input is a set of object files. 362 00:18:14,396 --> 00:18:16,020 And the output is going to be something 363 00:18:16,020 --> 00:18:18,990 that allows LD, the linker program, 364 00:18:18,990 --> 00:18:21,840 to know which symbol is located where 365 00:18:21,840 --> 00:18:23,540 in all the different modules. 366 00:18:23,540 --> 00:18:25,930 And if your linking succeeds, then it 367 00:18:25,930 --> 00:18:28,160 means that every symbol that's been referenced 368 00:18:28,160 --> 00:18:32,840 by any of the modules that's on the command line of this LD 369 00:18:32,840 --> 00:18:35,750 program, there are no undefined symbols remaining. 370 00:18:35,750 --> 00:18:38,850 So that's what symbol resolution does. 371 00:18:38,850 --> 00:18:41,620 And the input to this is really it looks something 372 00:18:41,620 --> 00:18:43,140 like you have a bunch of files. 373 00:18:43,140 --> 00:18:46,190 I'm going to call them F1 dot O, F2 dot O, all the way 374 00:18:46,190 --> 00:18:47,080 through FN dot O. 375 00:18:47,080 --> 00:18:49,826 OK, and for those of you who know what libraries, 376 00:18:49,826 --> 00:18:50,950 don't worry about them now. 377 00:18:50,950 --> 00:18:53,075 We are just going to take these individual modules, 378 00:18:53,075 --> 00:18:58,740 and we are going to produce out of it a single program that 379 00:18:58,740 --> 00:19:00,840 includes in it all of these different modules 380 00:19:00,840 --> 00:19:03,940 such that there are no unresolved symbols, OK? 381 00:19:03,940 --> 00:19:08,820 So, SQR is one file, and M dot C contains references 382 00:19:08,820 --> 00:19:11,960 SQR, we want to make it so the actual program, when 383 00:19:11,960 --> 00:19:15,810 the reference to SQR comes in, you know where it is. 384 00:19:15,810 --> 00:19:21,120 So, what each file contains when you do GCC on F1 dot CN, 385 00:19:21,120 --> 00:19:23,660 you get F1 dot O, it contains in it 386 00:19:23,660 --> 00:19:25,040 already a local symbol table. 387 00:19:25,040 --> 00:19:27,500 And you saw that in the example I showed you. 388 00:19:27,500 --> 00:19:30,320 And the symbol table has the following structure. 389 00:19:30,320 --> 00:19:34,740 There's two kinds of data in the symbol table on a local module. 390 00:19:34,740 --> 00:19:37,760 The first is a set of icons, D1, that 391 00:19:37,760 --> 00:19:42,800 correspond to global variables or functions that 392 00:19:42,800 --> 00:19:45,640 are defined in the module F1 dot C, 393 00:19:45,640 --> 00:19:48,780 and therefore are defined in the module F1 dot O. 394 00:19:48,780 --> 00:19:50,790 So, this just tells the linker, you 395 00:19:50,790 --> 00:19:54,860 know, if you see the symbol being invoked by somebody else, 396 00:19:54,860 --> 00:19:57,460 it's being defined in this module. 397 00:19:57,460 --> 00:19:59,690 OK, and it contains in it information 398 00:19:59,690 --> 00:20:01,420 about where in the local module it 399 00:20:01,420 --> 00:20:03,760 does it so that you know at what address it's 400 00:20:03,760 --> 00:20:06,130 being defined and so on. 401 00:20:06,130 --> 00:20:09,944 Likewise, you have here another set of symbols 402 00:20:09,944 --> 00:20:11,860 that you want which corresponds to things that 403 00:20:11,860 --> 00:20:14,510 are not defined in F1 dot O. 404 00:20:14,510 --> 00:20:21,880 And likewise, for each of these things you have D2 and U2, 405 00:20:21,880 --> 00:20:23,275 and so on, OK? 406 00:20:26,700 --> 00:20:29,210 So, the way the particular implementation 407 00:20:29,210 --> 00:20:31,756 of the linker under GNU-Linux works, 408 00:20:31,756 --> 00:20:33,630 and there are many ways to implement linkers, 409 00:20:33,630 --> 00:20:37,090 and there are very complicated linkers, and obviously simple 410 00:20:37,090 --> 00:20:38,410 linkers as well. 411 00:20:38,410 --> 00:20:41,010 The Linux one is particularly simple 412 00:20:41,010 --> 00:20:45,480 at least in terms of this way of linking, resolving 413 00:20:45,480 --> 00:20:46,670 all of the symbols here. 414 00:20:46,670 --> 00:20:48,890 You just scan the command line from left to right 415 00:20:48,890 --> 00:20:51,719 and you start building up three sets, OK? 416 00:20:51,719 --> 00:20:54,010 So we're going to build up three sets in this algorithm 417 00:20:54,010 --> 00:20:55,900 when we scan from left to right. 418 00:20:55,900 --> 00:20:59,090 The first set is a set I'm going to call O. 419 00:20:59,090 --> 00:21:02,580 And the idea in O is going to be all of object files 420 00:21:02,580 --> 00:21:04,170 that go into the program, that go 421 00:21:04,170 --> 00:21:05,690 into the output of the linking step. 422 00:21:05,690 --> 00:21:07,190 In this case, in the end, it's going 423 00:21:07,190 --> 00:21:08,398 to be pretty straightforward. 424 00:21:08,398 --> 00:21:11,369 It's going to be F1 dot O, union F2, all the way up to union FN. 425 00:21:11,369 --> 00:21:13,160 It's going to get a little more complicated 426 00:21:13,160 --> 00:21:14,480 when we talk about libraries. 427 00:21:14,480 --> 00:21:16,090 But for now, it's a very easy set. 428 00:21:16,090 --> 00:21:17,760 When you scan from left to right, 429 00:21:17,760 --> 00:21:20,010 you look at the next object module or the object file, 430 00:21:20,010 --> 00:21:29,700 and all you have to do is O goes to O union Fi dot O. 431 00:21:29,700 --> 00:21:32,430 OK, this is the I'th step of the algorithm. 432 00:21:32,430 --> 00:21:36,770 OK, now we have to go to the next step, which 433 00:21:36,770 --> 00:21:39,530 is get to the defined symbols. 434 00:21:39,530 --> 00:21:42,130 So as we go from left to right, we 435 00:21:42,130 --> 00:21:45,120 build up a set of defined symbols. 436 00:21:45,120 --> 00:21:47,640 Initially it's null, and then we start with D1, 437 00:21:47,640 --> 00:21:49,520 and then we do D1 union D2, and all the way. 438 00:21:49,520 --> 00:21:52,464 So up to the I'th step, what we're going to end up with is 439 00:21:52,464 --> 00:21:54,880 obviously at the I'th iteration, we are going to have some 440 00:21:54,880 --> 00:21:57,700 running set of defined symbols that have been defined 441 00:21:57,700 --> 00:21:59,100 in the modules so far. 442 00:21:59,100 --> 00:22:02,470 And we're just going to do D goes to D union Di where 443 00:22:02,470 --> 00:22:05,640 Di is the set of symbols that are defined in module number I. 444 00:22:05,640 --> 00:22:12,350 OK, in the last set, we're going to calculate, 445 00:22:12,350 --> 00:22:16,100 and the linker is going to calculate as a set U. 446 00:22:16,100 --> 00:22:19,730 And U is the set of all undefined symbols so far. 447 00:22:19,730 --> 00:22:21,940 So you have a set of undefined symbols. 448 00:22:21,940 --> 00:22:25,550 And you are at the I'th stage of this process where up to here 449 00:22:25,550 --> 00:22:28,037 you have a set U of undefined symbols. 450 00:22:28,037 --> 00:22:30,120 And now you're seeing this new module a bunch more 451 00:22:30,120 --> 00:22:31,420 undefined symbols. 452 00:22:31,420 --> 00:22:33,080 So clearly what you have to do first 453 00:22:33,080 --> 00:22:39,550 is do union Ui, correct, because you now have built up a bigger 454 00:22:39,550 --> 00:22:42,030 set of undefined symbols. 455 00:22:42,030 --> 00:22:44,660 But then, notice that there might be symbols that were 456 00:22:44,660 --> 00:22:47,290 previously undefined that are now being defined 457 00:22:47,290 --> 00:22:50,307 in the I'th module, right? 458 00:22:50,307 --> 00:22:52,140 Otherwise, I mean, if this set kept growing, 459 00:22:52,140 --> 00:22:53,310 then all you would end up with is 460 00:22:53,310 --> 00:22:54,750 a big undefined set at the end. 461 00:22:54,750 --> 00:22:56,500 So clearly there are symbols that are being defined 462 00:22:56,500 --> 00:22:57,680 and subsequent modules. 463 00:22:57,680 --> 00:22:59,460 So you have to subtract out a set. 464 00:22:59,460 --> 00:23:01,380 And the set you have to subtract obviously 465 00:23:01,380 --> 00:23:03,840 is the undefined set intersection 466 00:23:03,840 --> 00:23:07,960 whatever is being defined in this module. 467 00:23:07,960 --> 00:23:11,241 OK, and so the linker does this from left to right. 468 00:23:11,241 --> 00:23:12,740 It's actually a pretty simple linker 469 00:23:12,740 --> 00:23:14,531 because it doesn't go back and do anything. 470 00:23:14,531 --> 00:23:16,910 And it doesn't actually have to at least 471 00:23:16,910 --> 00:23:20,080 for this way of hooking up object files together. 472 00:23:20,080 --> 00:23:24,520 And what you get at the end are three sets: O, D, and U, OK? 473 00:23:24,520 --> 00:23:27,430 And, the linker outputs success, it then 474 00:23:27,430 --> 00:23:31,375 gets to the relocation stage of U is null. 475 00:23:31,375 --> 00:23:33,250 If there are no undefined symbols at the end, 476 00:23:33,250 --> 00:23:34,874 you know that now you can then relocate 477 00:23:34,874 --> 00:23:37,140 by modifying patching the different addresses together 478 00:23:37,140 --> 00:23:38,781 to produce your big program. 479 00:23:38,781 --> 00:23:40,280 But if the set is not null, you know 480 00:23:40,280 --> 00:23:41,870 that no matter what you do to relocate, 481 00:23:41,870 --> 00:23:43,380 there is some symbol that is not being defined, 482 00:23:43,380 --> 00:23:46,080 which means that module won't run, which means the program 483 00:23:46,080 --> 00:23:48,300 won't run, OK? 484 00:23:48,300 --> 00:23:51,205 So that's kind of what this linking program does. 485 00:23:51,205 --> 00:23:53,580 It just goes from left to right and resolves all symbols. 486 00:23:53,580 --> 00:23:55,832 And although it's done in the context of, 487 00:23:55,832 --> 00:23:58,290 we've discussed this in the context of a particular example 488 00:23:58,290 --> 00:24:00,860 of how GNU-Linux does its linking, 489 00:24:00,860 --> 00:24:02,090 the basic idea is the same. 490 00:24:02,090 --> 00:24:03,770 Essentially, all symbol resolution 491 00:24:03,770 --> 00:24:08,340 will turn out to use some variant of something that 492 00:24:08,340 --> 00:24:11,634 greatly resembles this method. 493 00:24:11,634 --> 00:24:13,050 There is actually only one problem 494 00:24:13,050 --> 00:24:17,230 with what I described so far, and it's wrong. 495 00:24:17,230 --> 00:24:19,961 So, what's the problem? 496 00:24:19,961 --> 00:24:20,460 Yeah? 497 00:24:29,170 --> 00:24:29,670 Right. 498 00:24:29,670 --> 00:24:31,860 If the symbol is undefined in F1 and defined in F2, 499 00:24:31,860 --> 00:24:34,950 you're fine because let's take this example. 500 00:24:34,950 --> 00:24:37,684 I had M dots it's OK. 501 00:24:37,684 --> 00:24:38,850 We're just building up sets. 502 00:24:38,850 --> 00:24:40,884 So you can have a symbol that's defined. 503 00:24:40,884 --> 00:24:42,300 You are saying, what if the symbol 504 00:24:42,300 --> 00:24:47,099 is defined in one of the files and undefined in the next file? 505 00:24:47,099 --> 00:24:48,390 No, it doesn't matter for this. 506 00:24:48,390 --> 00:24:50,240 It's going to matter when we talk about something called 507 00:24:50,240 --> 00:24:52,073 a library, but it's not going to matter here 508 00:24:52,073 --> 00:24:54,720 because we've built up these sets of what are defined 509 00:24:54,720 --> 00:24:55,840 and what are undefined? 510 00:24:55,840 --> 00:24:58,215 So let's say you defined a symbol here, and you come here 511 00:24:58,215 --> 00:25:00,680 and you find that it references a symbol that's 512 00:25:00,680 --> 00:25:04,710 not locally defined, but has been previously defined, right? 513 00:25:04,710 --> 00:25:06,410 That's been built up in the set, D. 514 00:25:06,410 --> 00:25:07,724 So, in the end -- 515 00:25:11,375 --> 00:25:13,000 Oh I see, so the code is a little wrong 516 00:25:13,000 --> 00:25:14,416 because I have to actually update. 517 00:25:18,929 --> 00:25:19,470 You're right. 518 00:25:19,470 --> 00:25:21,210 I had to modify that U line. 519 00:25:21,210 --> 00:25:21,841 This is wrong. 520 00:25:21,841 --> 00:25:22,340 Good. 521 00:25:22,340 --> 00:25:24,600 So I have to modify the set of undefined symbols 522 00:25:24,600 --> 00:25:27,010 to include the things that were previously defined. 523 00:25:27,010 --> 00:25:30,000 So actually it should probably have been U intersection D. 524 00:25:30,000 --> 00:25:35,731 And that will probably fix it. 525 00:25:38,330 --> 00:25:38,830 Good. 526 00:25:38,830 --> 00:25:42,281 Any other errors? 527 00:25:42,281 --> 00:25:42,780 Yeah? 528 00:25:42,780 --> 00:25:43,540 Right. 529 00:25:43,540 --> 00:25:44,350 So, there were two. 530 00:25:44,350 --> 00:25:45,720 This one I actually didn't realize. 531 00:25:45,720 --> 00:25:46,630 But it's actually right. 532 00:25:46,630 --> 00:25:47,660 So it should be U intersection D. 533 00:25:47,660 --> 00:25:49,260 The other one is that we just keep 534 00:25:49,260 --> 00:25:52,160 doing D goes to D union Di. 535 00:25:52,160 --> 00:25:56,060 But now, if I define a function, SQR here, 536 00:25:56,060 --> 00:25:58,800 and I define the same function SQR again here, 537 00:25:58,800 --> 00:26:01,720 we're not going to know which SQR to actually use. 538 00:26:01,720 --> 00:26:03,570 So we have a duplicate symbol. 539 00:26:03,570 --> 00:26:06,490 And so, in fact, what the algorithm actually 540 00:26:06,490 --> 00:26:08,980 ought to do is while it's doing this union computation, 541 00:26:08,980 --> 00:26:11,250 it better make sure that any symbol that's 542 00:26:11,250 --> 00:26:12,950 being defined in a subsequent module 543 00:26:12,950 --> 00:26:15,440 has not already been defined in a previous module. 544 00:26:15,440 --> 00:26:17,380 And, if there is a duplicate definition, 545 00:26:17,380 --> 00:26:19,680 it'll tell you that there's a duplicate definition. 546 00:26:19,680 --> 00:26:21,550 And I'm not going to show you an example, 547 00:26:21,550 --> 00:26:25,580 but if you just go and type out a little -- use SQR twice -- 548 00:26:25,580 --> 00:26:28,600 and you run the compiler, GCC, on it what you'll find is that 549 00:26:28,600 --> 00:26:31,220 it'll tell you that this symbol is multiply defined. 550 00:26:31,220 --> 00:26:34,380 OK, so we're not going to want that either. 551 00:26:34,380 --> 00:26:37,890 So once you obtain these different sets, 552 00:26:37,890 --> 00:26:43,000 what you'll end up with is information that 553 00:26:43,000 --> 00:26:44,992 will tell the linker everything about all 554 00:26:44,992 --> 00:26:47,450 of the different symbols, and where they have been defined, 555 00:26:47,450 --> 00:26:51,260 and in which object file they've been defined. 556 00:26:51,260 --> 00:26:53,830 And so, the step that it has to do after that 557 00:26:53,830 --> 00:26:55,410 is the relocation step. 558 00:26:55,410 --> 00:26:56,600 So this was the first step. 559 00:26:56,600 --> 00:27:07,250 It will turn out that this step is also pretty straightforward 560 00:27:07,250 --> 00:27:13,340 because what happens when GCC runs on a single C file 561 00:27:13,340 --> 00:27:15,550 and produces a single dot O file is 562 00:27:15,550 --> 00:27:18,670 it contains in it information on how to relocate 563 00:27:18,670 --> 00:27:20,780 all of the variables and all of the functions 564 00:27:20,780 --> 00:27:22,680 that are defined in that module. 565 00:27:22,680 --> 00:27:26,520 And there is a section of the object file 566 00:27:26,520 --> 00:27:29,380 that I haven't shown you called the relocation section that 567 00:27:29,380 --> 00:27:31,780 tells you, for any given variable in the text, 568 00:27:31,780 --> 00:27:33,710 or any given line of code in the text area, 569 00:27:33,710 --> 00:27:36,320 all of the load instructions and all of the variables, how 570 00:27:36,320 --> 00:27:38,296 they get remapped inside. 571 00:27:38,296 --> 00:27:39,170 So that's maintained. 572 00:27:39,170 --> 00:27:41,253 Basically the approach is to maintain local table. 573 00:27:44,880 --> 00:27:51,060 And, it's called the relocation section of your program, OK? 574 00:27:51,060 --> 00:27:53,290 So, we're going to get to loading in a little bit. 575 00:27:53,290 --> 00:27:56,410 But before we get to loading, I want to get back 576 00:27:56,410 --> 00:27:57,310 to symbol resolution. 577 00:27:57,310 --> 00:28:00,780 So here we just talked about taking a bunch of dot O files 578 00:28:00,780 --> 00:28:03,680 and producing a final program. 579 00:28:03,680 --> 00:28:07,510 But there is another notion of something called a library. 580 00:28:07,510 --> 00:28:10,040 So, how you do symbol resolution with a library? 581 00:28:15,230 --> 00:28:18,780 So an example of a library here is lib C dot A, which 582 00:28:18,780 --> 00:28:20,010 is the standard C library. 583 00:28:20,010 --> 00:28:21,540 And that's the library that contains 584 00:28:21,540 --> 00:28:22,960 the definition of printf. 585 00:28:22,960 --> 00:28:26,170 So if you use printf or any of these other standard functions, 586 00:28:26,170 --> 00:28:28,310 they define the C library. 587 00:28:28,310 --> 00:28:29,890 Another example is the math library, 588 00:28:29,890 --> 00:28:31,515 where you have the square root function 589 00:28:31,515 --> 00:28:33,632 and a bunch of other mathematical functions. 590 00:28:33,632 --> 00:28:35,090 So we are going to want to know how 591 00:28:35,090 --> 00:28:36,040 to resolve with the library. 592 00:28:36,040 --> 00:28:37,920 So first, we have to know what a library is. 593 00:28:37,920 --> 00:28:40,010 And what a library is, is just you 594 00:28:40,010 --> 00:28:43,017 take a bunch of object files together and basically just 595 00:28:43,017 --> 00:28:44,100 concatenate them together. 596 00:28:44,100 --> 00:28:47,550 It's not literally concatenation because the library also 597 00:28:47,550 --> 00:28:51,170 maintains an index that says, has information 598 00:28:51,170 --> 00:28:55,020 about what modules, what included 599 00:28:55,020 --> 00:28:57,480 dot O files contain which symbol definitions? 600 00:28:57,480 --> 00:29:00,350 But essentially it's just a concatenation 601 00:29:00,350 --> 00:29:04,385 of a bunch of dot O files, OK? 602 00:29:04,385 --> 00:29:06,010 And they get put together in a library. 603 00:29:06,010 --> 00:29:07,843 And there is some index information in front 604 00:29:07,843 --> 00:29:10,830 that says what is where inside the library. 605 00:29:10,830 --> 00:29:14,780 OK, so the approach to resolving symbols with a library 606 00:29:14,780 --> 00:29:17,180 is almost the same as what we have so far, 607 00:29:17,180 --> 00:29:18,760 except with one twist. 608 00:29:18,760 --> 00:29:20,730 And the twist is that often libraries 609 00:29:20,730 --> 00:29:23,630 are extremely big, much bigger than a single dot O file. 610 00:29:23,630 --> 00:29:26,620 And, one approach to resolving with libraries 611 00:29:26,620 --> 00:29:28,980 would be to apply the same approach here. 612 00:29:28,980 --> 00:29:30,590 So when you have something that's 613 00:29:30,590 --> 00:29:33,640 defined in the standard C library like lib C or the math 614 00:29:33,640 --> 00:29:35,160 library like lib L, you could just 615 00:29:35,160 --> 00:29:37,920 include the entire text of the library inside 616 00:29:37,920 --> 00:29:38,935 to build your program. 617 00:29:38,935 --> 00:29:40,810 But that just makes things extremely bloated. 618 00:29:40,810 --> 00:29:42,480 I mean, think about if you just wanted 619 00:29:42,480 --> 00:29:45,297 to use printf in your program like we had in this example 620 00:29:45,297 --> 00:29:47,380 and you had to include the entire C library, which 621 00:29:47,380 --> 00:29:50,270 is megabytes long, it seems like not a good way 622 00:29:50,270 --> 00:29:52,814 of doing the linking. 623 00:29:52,814 --> 00:29:55,230 So what we're going to do with the resolution of libraries 624 00:29:55,230 --> 00:30:00,130 is essentially the idea is to only include the dot O 625 00:30:00,130 --> 00:30:03,610 files that are in the library in which undefined symbols that 626 00:30:03,610 --> 00:30:07,390 were previously encountered are defined. 627 00:30:07,390 --> 00:30:10,010 And if you think a little bit about what I said, 628 00:30:10,010 --> 00:30:12,360 that's one reason why it usually a good idea 629 00:30:12,360 --> 00:30:18,030 when you do GCC and use the linker, use LD, 630 00:30:18,030 --> 00:30:20,170 to specify the libraries at the end 631 00:30:20,170 --> 00:30:22,194 because what we're going to do is 632 00:30:22,194 --> 00:30:23,860 we're going to take all the dot O files, 633 00:30:23,860 --> 00:30:25,901 and then they're going to be things like dash LM. 634 00:30:25,901 --> 00:30:27,890 I mean, the standard C library is usually, 635 00:30:27,890 --> 00:30:32,480 by default, already included on the command line. 636 00:30:32,480 --> 00:30:34,894 But if you have functions like square root and so on, 637 00:30:34,894 --> 00:30:37,310 here it what we're going to do is we're going to build up. 638 00:30:37,310 --> 00:30:38,380 What the linker is going to do is 639 00:30:38,380 --> 00:30:40,660 it is going to build up a set of undefined symbols 640 00:30:40,660 --> 00:30:42,094 until it gets to the end. 641 00:30:42,094 --> 00:30:44,260 And there's going to be undefined symbols remaining. 642 00:30:44,260 --> 00:30:45,850 If you use the square root program, 643 00:30:45,850 --> 00:30:47,500 the square root function, and you 644 00:30:47,500 --> 00:30:49,510 didn't write your own square root function, 645 00:30:49,510 --> 00:30:50,990 it's in the math library. 646 00:30:50,990 --> 00:30:53,447 Then you might ask for the math library to be included, 647 00:30:53,447 --> 00:30:55,530 and you want to pull the definition of square root 648 00:30:55,530 --> 00:30:59,860 from The math library the way in which the linker does 649 00:30:59,860 --> 00:31:04,780 to pull the right dot O file is that the math library is going 650 00:31:04,780 --> 00:31:08,360 to have information that says which object file contains what 651 00:31:08,360 --> 00:31:13,270 symbols, and anytime you see an undefined symbol at this stage, 652 00:31:13,270 --> 00:31:15,660 we are going to scan this archive looking 653 00:31:15,660 --> 00:31:17,990 for the symbol that's been undefined, in particular 654 00:31:17,990 --> 00:31:20,710 looking at this example for the square root symbol. 655 00:31:20,710 --> 00:31:24,110 OK, and when we find the square root symbol somewhere inside 656 00:31:24,110 --> 00:31:26,860 in some dot O file, we're going to take that dot O file, 657 00:31:26,860 --> 00:31:29,070 and not the rest of the library, and then 658 00:31:29,070 --> 00:31:30,910 use this algorithm that we did. 659 00:31:30,910 --> 00:31:34,900 So take that dot O file alone and push that as input 660 00:31:34,900 --> 00:31:37,110 to this algorithm. 661 00:31:37,110 --> 00:31:39,400 That's the way in which we're going to build it up. 662 00:31:39,400 --> 00:31:42,130 So that's why at least in this particular implementation 663 00:31:42,130 --> 00:31:46,240 of the linker which is very simple, if you put the LM 664 00:31:46,240 --> 00:31:48,260 way up in front, you are a little bit in trouble 665 00:31:48,260 --> 00:31:52,120 because if Fn dot O is the file that actually uses square root, 666 00:31:52,120 --> 00:31:54,510 and the math library was included well in front, 667 00:31:54,510 --> 00:31:56,790 then square root would not yet have 668 00:31:56,790 --> 00:32:02,090 been part of the undefined set of symbols until then. 669 00:32:02,090 --> 00:32:04,012 And there are other ways to deal with it. 670 00:32:04,012 --> 00:32:06,470 You can have linkers that go in more passes that presumably 671 00:32:06,470 --> 00:32:07,594 can deal with this problem. 672 00:32:07,594 --> 00:32:10,700 But this is just an example of how GNU-Linux does this, 673 00:32:10,700 --> 00:32:13,600 and that's just worth knowing. 674 00:32:13,600 --> 00:32:17,130 So this idea of linking and symbol resolution 675 00:32:17,130 --> 00:32:19,100 and relocation, people have worked 676 00:32:19,100 --> 00:32:21,232 on this for a very long time. 677 00:32:21,232 --> 00:32:22,940 And almost every software system uses it. 678 00:32:22,940 --> 00:32:26,750 In fact, if you use LaTeX to build your design papers and so 679 00:32:26,750 --> 00:32:29,754 on, it does symbol resolution as well 680 00:32:29,754 --> 00:32:31,670 because there is all sorts of cross-references 681 00:32:31,670 --> 00:32:32,700 that are there in LaTeX. 682 00:32:32,700 --> 00:32:34,908 And it uses essentially the same kinds of algorithms, 683 00:32:34,908 --> 00:32:37,451 go over the files and go over the documents 684 00:32:37,451 --> 00:32:39,200 in multiple passes and resolve the symbol. 685 00:32:39,200 --> 00:32:41,110 So it's a pretty general algorithm 686 00:32:41,110 --> 00:32:44,340 that we described here of building these different sets 687 00:32:44,340 --> 00:32:47,100 to ultimately figure out whether there are undefined references 688 00:32:47,100 --> 00:32:48,266 remaining at the end or not. 689 00:33:01,922 --> 00:33:03,380 So I want to step back for a minute 690 00:33:03,380 --> 00:33:06,530 and generalize on the different approaches 691 00:33:06,530 --> 00:33:10,790 that we've seen so far for doing symbol resolution. 692 00:33:10,790 --> 00:33:13,270 And today we saw one approach for doing symbol resolution. 693 00:33:13,270 --> 00:33:16,940 But in fact, last time we saw a couple of different approaches 694 00:33:16,940 --> 00:33:21,970 to resolve names whose values we didn't actually know. 695 00:33:21,970 --> 00:33:23,910 So, we're going to step back and generalize 696 00:33:23,910 --> 00:33:26,590 with a couple of different techniques 697 00:33:26,590 --> 00:33:31,470 that we saw the different examples. 698 00:33:31,470 --> 00:33:34,790 So the general problem here, the specifics 699 00:33:34,790 --> 00:33:38,350 are how you can find out where these undefined symbols are 700 00:33:38,350 --> 00:33:41,040 defined, or in the UNIX example, how 701 00:33:41,040 --> 00:33:45,580 you can take a big path name and identify which blocks contain 702 00:33:45,580 --> 00:33:48,660 the files or contain the data for the file that 703 00:33:48,660 --> 00:33:50,350 was named in the path. 704 00:33:50,350 --> 00:33:55,220 But the general problem is you have a set of names. 705 00:33:55,220 --> 00:33:59,230 And associated with each name is a value. 706 00:34:06,630 --> 00:34:09,179 And these names get associated with or mapped 707 00:34:09,179 --> 00:34:10,730 to the different values. 708 00:34:10,730 --> 00:34:12,989 In fact, the names get bound to the different values. 709 00:34:12,989 --> 00:34:15,513 And in this example, the linker needed 710 00:34:15,513 --> 00:34:17,429 to take a name like the definition of a symbol 711 00:34:17,429 --> 00:34:19,612 and needed to identify, what's the value associated 712 00:34:19,612 --> 00:34:20,320 with that symbol? 713 00:34:20,320 --> 00:34:22,389 The value here in this context is, 714 00:34:22,389 --> 00:34:25,701 where is this name, the square root function? 715 00:34:25,701 --> 00:34:26,909 Where is it actually defined? 716 00:34:26,909 --> 00:34:28,889 At what location is it? 717 00:34:28,889 --> 00:34:33,940 OK, and so the way in which that resolution 718 00:34:33,940 --> 00:34:36,524 is going to be done is done and general using something 719 00:34:36,524 --> 00:34:37,940 called the name mapping algorithm. 720 00:34:48,179 --> 00:34:52,600 And the mapping of a name to value being done by the name 721 00:34:52,600 --> 00:34:54,989 mapping algorithm takes into account, 722 00:34:54,989 --> 00:34:58,190 or takes as input something called the context. 723 00:34:58,190 --> 00:35:02,010 And I'll describe this with an example in a minute. 724 00:35:02,010 --> 00:35:05,100 So somebody has found a name to value. 725 00:35:05,100 --> 00:35:06,800 And when you are a linker or when 726 00:35:06,800 --> 00:35:08,887 you are part of the UNIX file system, 727 00:35:08,887 --> 00:35:10,970 and you're trying to identify the value associated 728 00:35:10,970 --> 00:35:13,550 with the name, you're going to have to resolve that name 729 00:35:13,550 --> 00:35:14,464 to find a value. 730 00:35:14,464 --> 00:35:15,880 And that resolution is going to be 731 00:35:15,880 --> 00:35:18,380 done in a particular context. 732 00:35:18,380 --> 00:35:20,200 So, for example, you might have two files 733 00:35:20,200 --> 00:35:22,200 with the same name in two different directories, 734 00:35:22,200 --> 00:35:23,460 and that's fine. 735 00:35:23,460 --> 00:35:26,520 The same name can have two different values 736 00:35:26,520 --> 00:35:29,800 because that resolution from the name to the correct value, 737 00:35:29,800 --> 00:35:32,420 in the directory case it's an inode number would 738 00:35:32,420 --> 00:35:34,720 be done in the context of the directory in which 739 00:35:34,720 --> 00:35:36,530 the resolution is being done. 740 00:35:36,530 --> 00:35:40,982 OK, so what we've seen over the last couple of days, 741 00:35:40,982 --> 00:35:43,190 the last lecture, and today, are three different ways 742 00:35:43,190 --> 00:35:44,000 of doing it. 743 00:35:44,000 --> 00:35:46,140 And it turns out that in almost every system, 744 00:35:46,140 --> 00:35:48,500 or in every system that I know of, anyway, 745 00:35:48,500 --> 00:35:52,810 there's basically three ways of doing this name resolution. 746 00:35:52,810 --> 00:35:56,320 The first way -- the simplest way -- is a symbol lookup. 747 00:35:56,320 --> 00:36:03,510 Look in the context of a dot O file, which 748 00:36:03,510 --> 00:36:05,740 has a set of defined symbols. 749 00:36:05,740 --> 00:36:08,130 Taking one of the symbols, the name, in that case, 750 00:36:08,130 --> 00:36:09,630 as input, and finding out where it's 751 00:36:09,630 --> 00:36:13,330 been defined in that dot O file is basically a table lookup. 752 00:36:13,330 --> 00:36:17,180 That's what that symbol table section describes. 753 00:36:17,180 --> 00:36:19,320 From the disk example from last time, 754 00:36:19,320 --> 00:36:21,580 the inode table is an example of a table lookup. 755 00:36:21,580 --> 00:36:23,550 I mean, there's a portion of disk 756 00:36:23,550 --> 00:36:27,362 that has in it the mapping between inode numbers 757 00:36:27,362 --> 00:36:28,570 and the corresponding inodes. 758 00:36:28,570 --> 00:36:29,670 And that's just a table lookup. 759 00:36:29,670 --> 00:36:32,003 So when you want to go from an inode number to an inode, 760 00:36:32,003 --> 00:36:33,310 you do a table lookup. 761 00:36:33,310 --> 00:36:35,580 And that's the simplest base form, 762 00:36:35,580 --> 00:36:40,340 base case of how this name resolution is done. 763 00:36:40,340 --> 00:36:42,800 The second way of doing name resolution 764 00:36:42,800 --> 00:36:44,550 is something called a pathname resolution. 765 00:36:47,790 --> 00:36:49,650 We didn't see an example of this today, 766 00:36:49,650 --> 00:36:51,720 but we saw an example of pathname resolution 767 00:36:51,720 --> 00:36:52,590 the last time. 768 00:36:52,590 --> 00:36:54,660 If you take a big UNIX pathname slash home 769 00:36:54,660 --> 00:36:57,940 slash foo slash bar, what we did was 770 00:36:57,940 --> 00:36:59,400 start left to right along that path 771 00:36:59,400 --> 00:37:05,760 and narrowed down our resolution of the file to get to the block 772 00:37:05,760 --> 00:37:10,610 that we wanted while going through a path, sorry, not 773 00:37:10,610 --> 00:37:15,520 a search path, but going to the path that names the item. 774 00:37:15,520 --> 00:37:19,200 And a third way of doing name resolution 775 00:37:19,200 --> 00:37:20,620 is what we saw today. 776 00:37:20,620 --> 00:37:24,180 And that's an example of searching through contexts. 777 00:37:29,104 --> 00:37:30,270 There's really no path here. 778 00:37:30,270 --> 00:37:32,190 I just tell you, here's a set of dot O files, 779 00:37:32,190 --> 00:37:33,440 and here's a set of libraries. 780 00:37:33,440 --> 00:37:36,030 And the symbols that are undefined 781 00:37:36,030 --> 00:37:38,310 are defined in different modules of my program, 782 00:37:38,310 --> 00:37:40,270 or F.O2 in different modules of my program 783 00:37:40,270 --> 00:37:42,580 are defined somewhere among these modules. 784 00:37:42,580 --> 00:37:44,770 And I'm not really going to tell the linker what's 785 00:37:44,770 --> 00:37:45,565 been defined where. 786 00:37:45,565 --> 00:37:47,190 It's up to the linker to figure it out, 787 00:37:47,190 --> 00:37:48,590 and it does that by basically running a search 788 00:37:48,590 --> 00:37:50,964 running a search through a variety of different contexts. 789 00:37:50,964 --> 00:37:53,210 So, each dot O file and each library 790 00:37:53,210 --> 00:37:56,920 is a new context in which a search for previously undefined 791 00:37:56,920 --> 00:37:57,805 symbol happens. 792 00:38:00,600 --> 00:38:04,320 And in this particular case, the search within each context 793 00:38:04,320 --> 00:38:07,650 takes the form of the table lookup, OK? 794 00:38:07,650 --> 00:38:10,010 So, those are the three techniques 795 00:38:10,010 --> 00:38:12,010 for how you do name resolution in general. 796 00:38:12,010 --> 00:38:13,510 And we saw two of them today, and we 797 00:38:13,510 --> 00:38:16,870 saw two of them, the first two, last time 798 00:38:16,870 --> 00:38:19,380 when we talked about the UNIX file system. 799 00:38:32,020 --> 00:38:36,180 So the last step in the process of what a linker does 800 00:38:36,180 --> 00:38:39,560 after symbol resolution and relocation, relocation 801 00:38:39,560 --> 00:38:42,942 in this particular kind of linking, I didn't use the term, 802 00:38:42,942 --> 00:38:44,650 but this form of linking is called static 803 00:38:44,650 --> 00:38:46,620 linking because we're going to take all 804 00:38:46,620 --> 00:38:49,559 of these different object files and defined on the command line 805 00:38:49,559 --> 00:38:51,100 or on the library, and build together 806 00:38:51,100 --> 00:38:55,137 a single big binary that has been linked once up front where 807 00:38:55,137 --> 00:38:55,970 the linker's called. 808 00:38:55,970 --> 00:38:57,210 That's called static linking. 809 00:38:57,210 --> 00:38:58,690 Internal relocation in that context 810 00:38:58,690 --> 00:39:00,450 is pretty straightforward. 811 00:39:00,450 --> 00:39:03,560 But so we're not going to talk more about that except 812 00:39:03,560 --> 00:39:05,960 to note that this relocation table is 813 00:39:05,960 --> 00:39:07,920 maintained in each object file. 814 00:39:07,920 --> 00:39:11,060 But the third step is a little more slightly more complicated, 815 00:39:11,060 --> 00:39:16,370 and actually varies a lot depending on the system. 816 00:39:16,370 --> 00:39:17,740 And that's program loading. 817 00:39:21,190 --> 00:39:23,700 And the problem that's solved by loading 818 00:39:23,700 --> 00:39:26,210 is that the linker produces an output 819 00:39:26,210 --> 00:39:27,369 program that's executable. 820 00:39:27,369 --> 00:39:28,660 It's and you can run a program. 821 00:39:28,660 --> 00:39:31,930 In UNIX, you run it by typing something on a command line. 822 00:39:31,930 --> 00:39:33,770 Or in Windows, you click on something, 823 00:39:33,770 --> 00:39:39,010 and effectively that causes a shell to execute a program. 824 00:39:39,010 --> 00:39:41,920 So somebody has to do the work of when 825 00:39:41,920 --> 00:39:44,030 you type something on the command line, somebody 826 00:39:44,030 --> 00:39:48,030 has to do the work of looking at what file has been typed, 827 00:39:48,030 --> 00:39:50,800 taking the contents of the file, loading it up 828 00:39:50,800 --> 00:39:53,590 into memory, passing control to something 829 00:39:53,590 --> 00:39:55,744 that can then start running the program. 830 00:39:55,744 --> 00:39:57,910 And typically the place where that control is passed 831 00:39:57,910 --> 00:40:01,820 is the interpreter corresponding to the program. 832 00:40:01,820 --> 00:40:03,520 So all of this work is done in UNIX 833 00:40:03,520 --> 00:40:04,740 by a program called execve. 834 00:40:04,740 --> 00:40:10,267 So the actual loader in UNIX is a program called execve. 835 00:40:10,267 --> 00:40:12,350 And its job, once you type it on the command line, 836 00:40:12,350 --> 00:40:13,850 is the shell invokes it. 837 00:40:13,850 --> 00:40:17,000 And what it does is to do what I said, which is look at the file 838 00:40:17,000 --> 00:40:19,760 name, take the contents of it, load it up into memory, 839 00:40:19,760 --> 00:40:25,620 and pass control to basically the first line of the program. 840 00:40:25,620 --> 00:40:28,005 Often, modern object files are a little more complicated. 841 00:40:28,005 --> 00:40:29,380 They actually have something that 842 00:40:29,380 --> 00:40:31,240 says who the interpreter of the program is. 843 00:40:31,240 --> 00:40:34,770 So you pass control to that interpreter, 844 00:40:34,770 --> 00:40:36,900 which in turn goes to a bunch of steps, 845 00:40:36,900 --> 00:40:39,370 and then invokes the first line in main. 846 00:40:39,370 --> 00:40:44,020 And that's what the C loader, the way in which 847 00:40:44,020 --> 00:40:46,540 loading a program that's written in C works. 848 00:40:50,550 --> 00:40:53,960 So, so far what we've seen is, as I mentioned, 849 00:40:53,960 --> 00:40:56,769 an example of linking called static linking where you take 850 00:40:56,769 --> 00:40:58,310 all these object files and libraries, 851 00:40:58,310 --> 00:41:00,143 and extract the right object files out of it 852 00:41:00,143 --> 00:41:01,600 and build a big program. 853 00:41:01,600 --> 00:41:05,760 So, what you'll find is that even small programs 854 00:41:05,760 --> 00:41:11,630 like this one, all it's doing is multiplying two numbers. 855 00:41:11,630 --> 00:41:23,080 If I compile it -- 856 00:41:23,080 --> 00:41:28,110 -- it's pretty big, almost 400 kB. 857 00:41:28,110 --> 00:41:29,610 I mean, it's multiplying two numbers 858 00:41:29,610 --> 00:41:32,290 and it takes 400 kB to multiply it. 859 00:41:32,290 --> 00:41:34,880 And the reason is that I made the mistake 860 00:41:34,880 --> 00:41:37,400 of including, I have to show the output to the user, 861 00:41:37,400 --> 00:41:38,940 so I call it printf. 862 00:41:38,940 --> 00:41:41,351 And printf happens with part of a big object file that 863 00:41:41,351 --> 00:41:42,600 defines a lot of other things. 864 00:41:42,600 --> 00:41:47,142 And that whole thing got built as part of the program. 865 00:41:47,142 --> 00:41:48,600 Now, if you have a lot of programs, 866 00:41:48,600 --> 00:41:50,266 so it's not just that the files are big. 867 00:41:50,266 --> 00:41:52,930 I mean, disks, as a mentioned a couple lectures ago, 868 00:41:52,930 --> 00:41:53,920 disks are cheap. 869 00:41:53,920 --> 00:41:56,250 And so that's not really the problem as much. 870 00:41:56,250 --> 00:41:58,830 The problem is that this is the entire program, 871 00:41:58,830 --> 00:42:00,290 so it has to get loaded in. 872 00:42:00,290 --> 00:42:04,270 So if you have some machine on which a hundred processors run, 873 00:42:04,270 --> 00:42:06,510 and each of those processors has printf's 874 00:42:06,510 --> 00:42:10,190 in a few places or even one printf, each of those programs 875 00:42:10,190 --> 00:42:13,760 is going to be extremely big. 876 00:42:13,760 --> 00:42:16,020 So the way in which you deal with this problem 877 00:42:16,020 --> 00:42:19,100 is to do something that's pretty obvious in retrospect. 878 00:42:19,100 --> 00:42:21,740 Why have multiple copies of the same module? 879 00:42:21,740 --> 00:42:25,920 Why don't we just have one copy of that module running and all 880 00:42:25,920 --> 00:42:28,270 the programs that use that module? 881 00:42:28,270 --> 00:42:30,950 And that's done using an idea called a shared 882 00:42:30,950 --> 00:42:37,420 object or shared modules. 883 00:42:37,420 --> 00:42:39,580 And this stuff is extremely hot. 884 00:42:39,580 --> 00:42:42,510 If you look at recent activity in almost every 885 00:42:42,510 --> 00:42:44,920 modular software system like you look at Apache 886 00:42:44,920 --> 00:42:47,200 or you look at many database systems, 887 00:42:47,200 --> 00:42:50,440 and also you look at GNU-Linux, there's been a ton of activity 888 00:42:50,440 --> 00:42:52,780 over the past five or six years on different ways 889 00:42:52,780 --> 00:42:55,544 of optimizing things so that, the idea is a very old idea. 890 00:42:55,544 --> 00:42:57,210 But there's still been a lot of activity 891 00:42:57,210 --> 00:42:59,170 making it efficient and practical 892 00:42:59,170 --> 00:43:02,450 over the last five to ten years. 893 00:43:02,450 --> 00:43:05,435 The problem with shared objects is the following. 894 00:43:05,435 --> 00:43:07,060 And this has become a little more clear 895 00:43:07,060 --> 00:43:11,930 when we talk about actually something called an address 896 00:43:11,930 --> 00:43:13,970 space a couple lectures from now. 897 00:43:13,970 --> 00:43:18,270 But the basic problem is that instructions are associated 898 00:43:18,270 --> 00:43:20,140 with memory locations, and they run 899 00:43:20,140 --> 00:43:21,990 each thing in the object file. 900 00:43:21,990 --> 00:43:24,670 And the binary has an instruction location. 901 00:43:24,670 --> 00:43:26,610 And data has a particular location 902 00:43:26,610 --> 00:43:28,610 associated with it as well. 903 00:43:28,610 --> 00:43:32,530 And the problem is that when you have two programs that each 904 00:43:32,530 --> 00:43:36,500 want to use a shared module, unless you're really careful 905 00:43:36,500 --> 00:43:39,190 about how you design it, it's going to be very hard for you 906 00:43:39,190 --> 00:43:41,240 to ensure that for both of those programs, 907 00:43:41,240 --> 00:43:44,080 this module that's going to be shared has exactly the same 908 00:43:44,080 --> 00:43:47,720 addresses because if you have in one program the module being 909 00:43:47,720 --> 00:43:53,170 called from address 17, and in another program the module is 910 00:43:53,170 --> 00:43:59,145 written up as 255, then that object cannot be shared, right? 911 00:43:59,145 --> 00:44:00,270 It's two different objects. 912 00:44:00,270 --> 00:44:02,103 And that's what happens with static linking. 913 00:44:02,103 --> 00:44:06,545 If you take the SQR.O module and you 914 00:44:06,545 --> 00:44:08,170 include that in two different programs, 915 00:44:08,170 --> 00:44:10,780 the addresses that get associated with it in the two 916 00:44:10,780 --> 00:44:13,590 different programs are going to be completely different. 917 00:44:13,590 --> 00:44:17,420 So one challenge, and a significant one in the shared 918 00:44:17,420 --> 00:44:21,240 object is objects that are shared 919 00:44:21,240 --> 00:44:24,260 by different programs running at the same time is 920 00:44:24,260 --> 00:44:29,670 to generate code that is what is called position independent. 921 00:44:29,670 --> 00:44:35,780 So it doesn't have in it anything 922 00:44:35,780 --> 00:44:38,550 that's different for the different programs. 923 00:44:38,550 --> 00:44:43,380 And so this is called position independent code. 924 00:44:43,380 --> 00:44:46,550 So the idea is when you call the module 925 00:44:46,550 --> 00:44:48,460 on your computer, the program counter 926 00:44:48,460 --> 00:44:49,879 is going to point to something. 927 00:44:49,879 --> 00:44:51,670 And all of the addresses inside that module 928 00:44:51,670 --> 00:44:53,269 are going to have addresses. 929 00:44:53,269 --> 00:44:55,060 Obviously, they're going to have addresses, 930 00:44:55,060 --> 00:44:58,280 but they're going to be relative to the program counter. 931 00:44:58,280 --> 00:45:01,356 So when you jump to a location, I'm going to jump to 317. 932 00:45:01,356 --> 00:45:02,730 You're going to jump to something 933 00:45:02,730 --> 00:45:06,020 that says five locations from where you are now. 934 00:45:06,020 --> 00:45:08,680 So it's a kind of addressing called PC relative addressing. 935 00:45:08,680 --> 00:45:10,930 And that's not going to be the only thing that's used. 936 00:45:10,930 --> 00:45:14,470 But by and large, a requirement for position independent code 937 00:45:14,470 --> 00:45:16,707 is that all of the addressing be relative to, say, 938 00:45:16,707 --> 00:45:17,540 the program counter. 939 00:45:20,880 --> 00:45:26,450 And once you have this kind of position independent code, what 940 00:45:26,450 --> 00:45:29,000 you have to do in your program, if you're using something 941 00:45:29,000 --> 00:45:31,360 like the square root program, let's say, 942 00:45:31,360 --> 00:45:33,640 OK, in the math library when you do the linking, 943 00:45:33,640 --> 00:45:35,990 you don't have to include the object file in which 944 00:45:35,990 --> 00:45:38,710 square root is defined. 945 00:45:38,710 --> 00:45:40,370 All that your object file has to do now 946 00:45:40,370 --> 00:45:42,940 is at the time it was linked, there 947 00:45:42,940 --> 00:45:46,520 is some library, runtime library, 948 00:45:46,520 --> 00:45:50,100 that you know when you link the program has 949 00:45:50,100 --> 00:45:51,950 the definition of square root. 950 00:45:51,950 --> 00:45:54,150 So what the program contains when 951 00:45:54,150 --> 00:45:59,070 you run this F1 dot O, F2 dot O, all the way through the library 952 00:45:59,070 --> 00:46:02,160 is it's just going to maintain a pointer, a name actually. 953 00:46:02,160 --> 00:46:04,690 It's going to maintain a name to where the square root 954 00:46:04,690 --> 00:46:05,620 function is defined. 955 00:46:05,620 --> 00:46:07,420 It's going to be a filename. 956 00:46:07,420 --> 00:46:11,204 OK, and so this is why sometimes when you type in a program, 957 00:46:11,204 --> 00:46:13,620 and somebody has changed the configuration on your machine 958 00:46:13,620 --> 00:46:15,650 and built before, and somebody changes the configuration, 959 00:46:15,650 --> 00:46:17,816 sometimes you get an error message that while you're 960 00:46:17,816 --> 00:46:20,430 running the program, you get an error message that says 961 00:46:20,430 --> 00:46:23,040 some library dot SO not found. 962 00:46:23,040 --> 00:46:24,910 It worked two days ago. 963 00:46:24,910 --> 00:46:25,970 It doesn't work now. 964 00:46:25,970 --> 00:46:27,480 And the reason for that is somebody 965 00:46:27,480 --> 00:46:30,046 may have moved things around and you get an error 966 00:46:30,046 --> 00:46:31,420 not when you compile the program, 967 00:46:31,420 --> 00:46:32,660 but when you run the program. 968 00:46:32,660 --> 00:46:34,460 And many executions of program may not actually 969 00:46:34,460 --> 00:46:35,660 trigger the error at all. 970 00:46:35,660 --> 00:46:39,104 It may get triggered only when the actual object is needed. 971 00:46:39,104 --> 00:46:40,520 And that's called dynamic linking. 972 00:46:44,480 --> 00:46:46,740 And again, the way in which we do dynamic linking 973 00:46:46,740 --> 00:46:48,890 is to use the same name and constants. 974 00:46:48,890 --> 00:46:52,470 Rather than embed the entire object file corresponding 975 00:46:52,470 --> 00:46:57,190 to the module corresponding to a name of a function that's 976 00:46:57,190 --> 00:47:00,880 being defined elsewhere, maintain a reference to it. 977 00:47:00,880 --> 00:47:03,650 And load that reference up at runtime. 978 00:47:03,650 --> 00:47:05,780 Now, in order to enable for that object 979 00:47:05,780 --> 00:47:07,530 to be shared between different programs, 980 00:47:07,530 --> 00:47:10,520 all of the addressing inside that has to be relative. 981 00:47:10,520 --> 00:47:13,540 That is, it has to be independent of what 982 00:47:13,540 --> 00:47:17,360 the PC's value for the starting point of that module is. 983 00:47:17,360 --> 00:47:21,010 And that's called position independent code. 984 00:47:21,010 --> 00:47:23,120 So that's the basic story behind how 985 00:47:23,120 --> 00:47:26,880 linking works and almost all software systems 986 00:47:26,880 --> 00:47:30,710 involving libraries and modules ends up having a linking in it. 987 00:47:30,710 --> 00:47:33,000 I mentioned LaTeX as an example before. 988 00:47:33,000 --> 00:47:34,809 Even document systems have linking in it. 989 00:47:34,809 --> 00:47:37,350 And it's a pretty fundamental algorithm that we talked about. 990 00:47:37,350 --> 00:47:39,610 It's specific to GNU-Linux, but the basic idea 991 00:47:39,610 --> 00:47:41,110 is pretty common. 992 00:47:41,110 --> 00:47:43,950 Now, what we'll see next time is that this way of modularizing 993 00:47:43,950 --> 00:47:45,450 has a lot of nice properties, allows 994 00:47:45,450 --> 00:47:47,630 you to build big programs, but at the same time 995 00:47:47,630 --> 00:47:50,230 has pretty bad fault isolation properties that we'll 996 00:47:50,230 --> 00:47:52,490 talk about next time.