Hamming Art Science Engineering
The Art of Doing Science and Engineering by Richard Hamming (Amazon, Goodreads) is an excellent book. It distills a career of scientific research and engineering practice into a series of focused chapters, each with a specific theme.
Some of the examples of views are a little outdated, but this is principally because Hamming did his work in the mid-20th century. While a lot has changed since then, it is testament to his skill and good-opinions that most of what he wrote is still valuable now.
I read the book on my ReMarkable, making highlights as I went. I typed up a subset of what I highlighted below, mainly for my own reference.
- p2: Education is what, when and why to do things. Training is how to do it.
- p3: The number of scientists alive has been exponential. 90% of scientists who ever lived are now alive.
- p4: What do you focus on in your studies? Focus on the fundamentals, or what you think are the fundamentals.
- p4: Develop the ability to learn new fields of knowledge when they arise so you won’t be left behind.
- p5: Fundamentals are things that have lasted a long time (similar to Lindy). Another way to detect them is to see if they can be derived using the standard methods in the field.
- p6: The future will be more limited by human organization than it will be by
- NOTE: This is why its important that we make governance cool again. We must invest into having good governance (of technology) to minimize the rate at which technology development outstrips the capacity for (good) policy creation.
- p6/7: Having a good vision of what you want to achieve will help you go further. Having no vision is essentially like walking in a random direction from a starting point. The accuracy of the vision is less important than having a vision at all. There are many paths to greatness, and you only need one.
- p7: The standard approach of breaking knowledge down into
disciplines/sub-disciplines has two drawbacks. First, it conceals the
underlying homogeneity of knowledge. Second, it leaves out the important
things that fall between the disciplines.
- NOTE: Those in-between parts are what you mine when you truly become interdisciplinary.
- p8: You have only one life - you should try to make significant contributions to humanity rather than just get through comfortably.
- p13: You must understand the deep theories of things to be effective far into the future as the higher-level ideas slowly advance
- p13: You should pay attention to what you know. Be able to re-think it from the ground up and decide for yourself whether it is still applicable. As the world changes, theories lose their relevance or start to look outdated in the face of new information.
- p15: New innovation often takes the form of an “S” curve. Often innovation overall will be successive “S” curves overlaid onto one another - as one invention takes over from another. This means that you see times of high innovation, then plateaus of little innovation.
- p27: Computers are not just number crunchers. They are symbol manipulators.
- p27: Almost everybody who opens up a new field doesn’t understand it in the same way that followers do. This is because creators have to fight through many dark difficulties, misunderstanding and confusion. They are the ones exploring the dead ends and finding the way. The people who come after already have a path to follow, which is far more clear and easy.
- p31: Programming is kinda like novel writing in the sense that it is very imaginative.
- p31: You need to think carefully before programming. Think about the whole thing, including testing and how maintenance will happen.
- p32: The most productive programmers are far more productive than the least productive. Hamming suggests firing the least productive, but paying the most productive ones very well.
- p33: When giving a talk, consider what your audience wants to hear. Do not give dull talks, but walk the line between science communication and entertainment.
- p34: Communicate your excitement to the audience. Don’t relax too much when giving your talks otherwise your audience will fall asleep.
- p37: When you use a computer to do a job, it never does the same job as if somebody was to do it manually. Rather it does an equivalent job.
- p38: The most important things you can do are those which make a difference to your organization (whatever it is). 10% improvements are fine, but keep an eye out for the great new things on the horizon.
- p40: Computers manipulate symbols, but not information.
- p41: We must consider, “out of all of life’s burdens, which are those machines can relieve, or significantly ease, for us?”
- p42: It’s hard to define “thinking”. Turing sidestepped the question with his Turing Test, since he specified a way to recognize thinking, but didn’t define thinking directly. I.e. a “I know it when I see it” thing.
- p42: Thinking might not be a yes/no thing, but rather a matter of degree.
- p50: We should focus on AI where humans and machines can work together, not where they compete between each other. Robots will displace humans from doing menial tasks, but this should only allow humans to do humane jobs.
- p52: We originally started programming computers to play games in order to gain insights into how the human brain works. Eventually this motivation changed to trying to make computers win games. The original aim is still likely most important.
- (whole chapter) We should carefully consider the `solution space' of any given design problem. Consciously probe this space for novel solutions.
- p67: The meaning of any symbol depends on how it is processed. Symbols have no inherent meaning. For example, the meaning of an instruction is dependent on the subroutine (i.e. context) it appears in.
- p81: One of Hamming’s greatest discoveries (error correcting codes) occurred to hum while there was a great deal of emotional stress. This is a characteristic of most great discoveries. Working calmly will let you elaborate and extend things, but breakthroughs generally come from great frustration and emotional involvement.
- p83: Hamming was able to invent error correcting codes because he was familiar with the binary system which wasn’t common in the late 40’s (then, people used analog computers). By understanding this (then) niche field, he was able to come up with new inventions which intersected that field with more established areas of study.
- p87: Pasteur’s rule: Luck favors the prepared mind. You succeed by preparing yourself to succeed. Yes, luck exists. You need to have an element of luck to be able to discover new things, but in Hamming’s career, there were many people who were in a similar position to him and who did not make great discoveries. Hamming was lucky, but he was also preparing himself to understand what was going on, more than the other people around who were merely reacting to things, and not thinking deeply as to what was behind the surface phenomena.
- p87: Life presents you with lots of opportunities to do great things. Prepared people will hit one or more successes, but unprepared people will miss almost every time.
- p88: What it takes to be great in one time period is different from what it takes to be great in the next. You can’t simply copy the approaches of others (e.g. Hamming), but rather think for yourself about the nature of the future that you will live in, and adapt your approach with that in mind.
- p96: Inspect the definitions of terms and other things. To what extent do results stem from the definition framed to get a desired result? How often were definitions framed under one condition, but applied under another? For the first question, think IQ tests - they are defined to give a normal distribution of IQ among the population. They could just as easily be defined to give an exponential distribution of IQ. For the second question, think economics. Much of economic theory rests on assumptions that do not hold (or rarely hold) in practice.
- p98: Learning a new subject is something you will have to do many times in your career if you are to be a leader and not left behind as a follower by newer developments.
- p98: When you see something weird, always ask yourself; “What is really going on here?” Work from first-assumptions where possible.
- p107: People always want to think that something new is just like the past. They want to be comfortable with the new stuff. As such, they prevent themselves from making significant contribution to the new field being created right under their noses.
- p107: When something is claimed to be new, don’t quickly think that it is just the past slightly improved. Rather, it may be a great opportunity for you to do significant things. Then again, it may indeed be nothing new.
- p114: If you want stuff to be named after you, then it must be your friends that make you famous by quoting and citing you. As such, it pays to be helpful to others as they try to do their work. They may in the future give you credit for your work, which is far better than trying to claim it for yourself.
- p114: Cooperation is essential because we work on such complex projects. You can no-longer work alone as an individual on most stuff. Instead, learn to work effectively in a team and seek out places where you can help others. In any case, the fun of working with other people on important problems is more pleasurable than the resulting fame.
- p114: You should be careful when writing up results that you worked on with others - they may see your contributions and theirs in a different light to how you do. If they write up results, then they can cite you as a co-author as they wish.
- p119: When something can’t be done remember the reason why. Then later on, when the circumstances have changed, you will recognize that maybe it can actually be done. Review the /details/ of why the task could not originally be done, and ascertain whether it is still the case.
- p125: To the extent that you can choose, work on problems that you think are important
- p126: As people get more specialized and narrower in their knowledge, we need more people to keep the larger view and see that things are done honestly. These are the people who can ensure that a broader vision is followed, and link separate areas of knowledge together.
- p127: If you don’t occasionally doubt accepted rules, then you are unlikely to be a (thought) leader. At the same time, if you doubt too much, then you will be paralyzed and do nothing. How to strike a balance between these two extremes is a matter of style - you must learn and develop it for yourself. Big advances usually come from changes in the underlying assumptions of a field.
- p131: When running simulations, it is better to run a few simulations and understand what happened well than it is to run hundreds and use trial and error. The former way allows you to get an intimate feeling about the simulation.
- p131: Doing simple simulations at the early stages of research lets you develop insights into the whole system which would be disguised in a full-scale simulation. Therefore, where possible, start with a simple simulation and then evolve it into a more complete and accurate simulation later.
- p137: Older minds generally have more trouble adjusting to new ideas than younger minds. This is relevant, since you will often be presenting to older people throughout your career, so bear this in mind.
- p155: You should try to keep abreast of new developments in your field by actively anticipating the way things and ideas might go, then seeing what actually happens. Your anticipation means that you are far better prepared to absorb new things when they arise than if you sit passively and merely follow the progress. “Luck favors the prepared mind”.
- p156: What is technologically feasible and even economically better is often constrained by legal, social and economic conditions. Bear this in mind.
- p160: “What you learn from others you can use to follow. What you learn for yourself you can use to lead."
- p161: You must learn to recognize different concepts in different forms. For example in computer programming, there are many different problems that are suitable for solving through recursion. You must be able to recognize abstract ideas and apply them in specific contexts.
- p169: Mathematics is a sort of a universal mental tool for clear thinking.
- p169: Part of what makes mathematics effective is that there are analogies between the equations on our desks and the real world. Only while these analogies are accurate, can mathematics have predictive power.
- p169: Hamming theorizes that in the past, we have found the ‘easy’ applications of maths where there is a close correspondence between the underlying structure of maths and the corresponding real-world phenomena. In time, this won’t be the case, and we will have to be satisfied with maths having a looser relationship with the real world - it’ll be simply too hard to model. As such, we may have to have many different models where the whole is more than the sum of the parts.
- p172: There can be multiple theories that can account for the same observations and agree on all the predicted details. You can’t go from a body of data to a unique theory.
- p173: Man is not a rational animal. He is a rationalizing animal.
- p175: Hamming predicts that in the future, we won’t be able to fully “understand” things because they will be too complex/complicated. He implies that we will invent new tools that can cope with this.
- p177: Creativity seems, to be the ability to “usefully” put things together which were not perceived to be related before. It might be the initial psychological distance between things that counts the most.
- p177: Creativity in an individual can be improved/developed. You must develop your own effective style in order to have “great ideas”.
- p177: “I often suspect creativity is like sex; a young lad can read all the books you have on the topic, but without direct experience he will have little chance of understanding what sex is - but even with experience, he may still not understand what is going on!”
- p178: Creativity tends to have the following pattern. First there is a recognition of the problem at hand, at least in a dim sense. Then the problem is refined over some period of time (you shouldn’t be too hasty here - put the problem in a conventional form and you will get a conventional (uncreative) solution). This second stage requires emotional involvement, which is your commitment to finding a fundamental and novel solution. Either you come up wit ha solution or otherwise temporarily abandon the problem. Temporary abandonment is a common feature of great creative acts, often it is required to enable the subconscious to find a new approach. Finally, there is a moment of insight where you can see the solution. Of course, you might be wrong, where on closer examination, your solution is faulty. Even then, it might be able to be saved with a revision.
- p178: When stuck at work, Hamming asks himself “If I had a solution, what would it look like?”. This tends to sharpen the approach because it may reveal new ways of looking at the problem that might have been initially ignored.
- p178: Once you have a solution, there is still a lot more work to be done. It needs to be cleaned up and organized so taht others can see it. The public presentation to others may require you to reframe the solution in a more standard, less idiosyncratic way than you first conceived it. Revising the solution often brings clarity in the long run.
- p178: How to bet your subconscious to think of stuff? Saturate it by only thinking of that thing for hours/days even weeks. Left only with a single problem, it will mull it over. Then, the solution will pop into our mind one day.
- p178: Sometimes analogy is the most important tool for creativity, where
something seems like something else. This means that being widely acquainted
with many fields is helpful, providing that you can recall the knowledge when
needed (rather than only realizing when led directly to it). This flexible
access to knowledge comes from looking at knowledge from many different angles
while you are acquiring it, and turning the idea over to see all of its sides
before ‘filing it away’.
- NOTE: This is why books that have a single idea are useful despite explaining one thing in 200+ pages. They look at the idea from lots of different angles and give you a chance to really absorb it.
- p179: Getting to the fundamentals of the field involves looking at it from many angles, then deciding what is fundamental and what is frills.
- p179: We reason mainly by analogy.
- p179: When learning new things, try to create ‘hooks’ for that knowledge so you’ll be able to recall it in the future. One way to do that is to consider ways in which the knowledge might be applicable in the future - i.e. new applications.
- p179: Without self-confidence, you are not likely to create great, new things. There is a thin line between having enough self-confidence and being over-confident.
- p179: We are, in a very real sense, the sum total of our habits. By changing our habits, we are changing ourselves. In doing so, we can change the path along which we want to go.
- p180: If you are to succeed, you must be creative in the face of rapidly changing technology which will dominate you career. Society won’t stand still for you, rather , it will evolve more and more rapidly as technology plays an increasing role everywhere.
- p180: You must know when to drop the wrong problem. If you can’t drop a problem then you will be stuck with it for the rest of your career (this happened to Einstein in his mid/late years).
- p180: With very creative people, their previous successes convince them that they can solve any problem. However, it often gets harder to solve highly creative problems with advancing age in some fields. These include maths, physics, etc. Yet in literature, composition, statesmanship etc age is an asset.
- p180: If you want to do significant things, now is the time to start thinking! Don’t wait until the proper moment - it may never arrive!q
- p181: As our knowledge grows exponentially, we cope with the growth mainly through specialization.
- p181: An expert is one who knows everything about nothing; a generalist knows nothing about everything.
- p181: “In an argument between a specialist and a generalist the expert usually wins by simply: (1) using unintelligible jargon, and (2) citing their specialist results which are often completely irrelevant to the discussion. The expert is, therefore, a potent factor to be reckoned with in our society. Since experts are both necessary, and also at times do great harm in blocking significant progress, they need to be examined closely.”
- p181: Kuhn defined the concept of a ‘paradigm’. Most of the time, any
particular science has accepted a given set of assumptions which are not often
mentioned or discussed. The teaching of new students is predicated on these
assumptions, who accept them without being aware of how extensive they are.
Workers in the field then elaborate on them, ignoring contradictions which
appear. Eventually, there will be a sudden change in the paradigm, which
creates new answers to old questions and brings new questions in itself.
- NOTE: This is the case with the Von Neumann machine in Computer Science/Engineering (starting to be superseded more and more), and with neoclassical assumptions in economics/competition law.
- p181: Change is usually resisted by the establishment, which has put lots of effort into the old approach. Usually, the new eventually trumps the old.
- p182: All proofs of impossibility rest on a number of assumptions, which might not apply in any given situation.
- p182: New ideas seldom come from experts in a field. You can’t blame them because it is more economical to use old and successful ways of thinking before trying new ways.
- p182: “If an expert says something can be done he is probably correct, but if he says it is impossible then consider getting another opinion.”
- p184: Experts have a dilemma, there are many crackpots outside the field who are crazy and only a few with good ideas. They most of the time, will simply decide not to listen to outsiders.
- p184: If you are an expert, you should ask yourself “What would I accept as evidence to show I’m wrong?” and “Why do I believe whatever I do?”.
- p186: You need to keep up with what’s going on in your field or you’ll be overtaken. If you don’t keep up, you’ll eventually drop out and disappear.
- p186: “Civilization is merely a thin veneer we have put on top of our anciently derived instincts, but the veneer is what makes it possible for modern society to operate. Being civilized means, among other things, stopping your immediate response to a situation, and thinking whether it is or is not the appropriate thing to do.”
- p187: There is never time to do the job right, but always time to fix it later (Hamming doesn’t like this).
- p191: Lots of economic indicators are incredibly unreliable. They come from when societies were manufacturing societies, but now many/most are service societies. For example unemployment figures don’t distinguish between the unemployed and unemployable.
- p191: Institutions, like people, tend to move only when forced to.
- p193: Averages are meaningful for homogeneous groups (with respect to actions that later may be taken), but meaningless for diverse groups. For example, the average adult has one testicle and the average number of children people have is likely a fraction.
- p193: If you want to find out data for something that people would be embarrassed or reluctant to tell you (e.g. whether they are cheating on their partner), then you can use the following method (n.b. Hamming has not tested this). You interview people and toss a coin without anybody but themselves seeing the outcome. If it is heads then they should claim to be cheating, if it is tails, then they should tell the truth. Thus, each person individually has anonymity (or plausible deniability), but with a large sample size, you could find the incidence of cheating.
- p195: Parables are often more effective than statements.
- p195: “A man was examining the construction of a cathedral. He asked a stone mason what he was doing chipping the stones, and the mason replied, “I am making stones”. He asked a stone carver what he was doing, “I am carving a gargoyle”. And so it went, each person said in detail what they were doing. Finally he came to an old woman who was sweeping the ground. She said, “I am helping build a cathedral”.”
- p195: Very few professors would say “I am educating students to prepare them for their future careers” when asked a similar question.
- p195: Most of the time, we are so immersed in the details of one part of the whole, that we do not think of how and what we are doing related to the larger picture.
- p195: Systems engineering is attempting to keep the larger goals in mind all the time, and translate local action into global results.
- p196: The first rule of systems engineering is: If you optimize the
components, you will probably ruin the system performance
- NOTE: Taleb talks about this in Antifragile. He says that systems where each component is optimized are fragile, but systems that degenerate and have redundancy are antifragile.
- p197: Few people set out to reform a system by first trying to find the total system problem. Instead they attack the first symptom they see. This isn’t effective.
- p198: Part of systems engineering design is to prepare for changes so they can
be gracefully made and still not degrade the other parts
- NOTE: This is a core principle of the current software engineering paradigm that I understand very well, but it is probably not apparent some people who don’t come from this background. At the least, they may lack a practical understanding how it can be achieved.
- p198: The half life of engineering details is 15 years, and is probably decreasing slowly.
- p199: Good design generally includes the graceful decay of performance when the specifications are exceeded.
- p199: Specialists brought together to make a team are the basis of systems
engineering. But, between jobs they must go back to their specialties to
maintain their expertise. Usinga group too often to fight fires is detrimental
in the long run since the individuals will start to lose their expert skills.
- NOTE: This is a manager’s view of the fact that if you don’t use a skill then you’ll lose it.
- p200: The heart of systems engineering: acceptance that there is neither a definite fixed problem nor a final solution. Evolution is the natural state of affairs.
- p202: The way you choose to measure something controls what measurements you will get.
- p202: Accuracy of measurement tends to get confused with the relevance of measurement, much more than most people believe. For example, at school it is easy to measure training but hard to measure education. You tend to see final exams which measure training but neglect the education part.
- p204: “It is very similar in Computer Science where the ability to cope with the mass of programming details favors one kind of mind, one which is often negatively correlated with seeing the bigger picture.” > interesting…
- p206: The popularity of a measure has little to do with its accuracy or relevance
- p209: We only have one life to lead. It’s better to do significant things than
plan to ‘just’ get along through life. Certainly, near the end of life it’s
nice to look back at what you’ve accomplished (rather than just a life where
you have merely survived and amused yourself).
- Note: I’m guessing Hamming wasn’t an Epicurian then!
- p209: It is worth trying to accomplish the goals you set for yourself, and it is worth having high goals
- p209: You should always try to do first-class work.
- p209: One reason that people don’t set themselves high goals is because they think achieving them is down to luck. But remember what Pasteur said - “luck favors the prepared mind”. There is an element of luck, but to a large extent, things are also up to you.
- p210: Newton apparently observed that if others would think as hard as he did, then they would be able to do the same things as he. Edison said that genius was 99% perspiration and 1% inspiration. Hard work applied for long years leads to the creative act. It is rarely just handed to you without any serious effort.
- p210: Brains are nice, but many people with not great IQ’s have done great things.
- p210: Among the important properties to possess is the belief that you can do important things. If you do not work on important problems, then how can you expect to do important work?
- p211: Confidence in yourself is an essential property. You could also call it “courage”. You can develop this in yourself. Look at your successes and pay less attention to your failures. Courage is important because research of ten has long periods with no success and many discouragements.
- p211: The desire for excellence is an essential feature for doing great work. If you have a great vision and a desire to do significant work, then you’re on the right track.
- p211: Often, the most famous people think that they can only work on important
problems. Thus they fail to plant the little acorns which grow into mighty oak
trees. You should work on small things which seem to you to have the
possibility of future growth.
- Note: This is similar to what Taleb says about research funding (link). Planting the little acorns is essentially taking advantage of the convexity of the payoffs.
- p211: Working with your (hypothetical) office door closed lets you get more work done per year than if you had an open door. However, those with closed doors will work on slightly the wrong problems while those who leave their doors open will get more less done but will work on the right problems.
- p212: The conditions you want are seldom the best ones. But the interaction with reality tends to push you towards significant discoveries which you might not have thought about while doing research without these constraints.
- p212: Great people have a great deal of drive to do things.
- p212: I had worked with John Tukey for some years before I found he was essentially my age, so I went to our mutual boss and asked him, “How can anyone my age know as much as John Tukey does?” He leaned back, grinned, and said, “You would be surprised how much you would know if you had worked as hard as he has for as many years”
- p212: Intellectual investment is like compound interest. The more you do, the more you learn how to do, so the more you do, etc. Hamming estimates the compounding is well over 6% - one hour extra per day over a whole lifetime will more than double the total output.
- p212: But be careful, it’s not about who works the hardest. You need to be working on the right problem, at the right time, in the right way. This is your style.
- p212: Hamming would set aside Friday afternoons for “great thoughts”. After lunchtime, he would only think about the big questions - where was CS going, what was the role of computers in Bell Telephone Labs, etc. He strongly recommends taking this time on a regular basis.
- p213: Another great trait to have is tolerating ambiguity - being able to believe and disbelieve at the same time. For example, you need to believe that your field of research is the best, but also believe that there is room for improvement.
- p213: Most great people have 10-20 problems that they regard as basic and of great importance, but don’t know how to solve. They keep them in their mind, hoping to get a clue as to how to solve them.
- p213: “I have long held the attitude of telling every one freely my ideas, and in my long career I have only had one important idea ‘stolen’ by another person. I have found people are remarkably honest if you are in your turn.”
- p214: To sell ideas, you must do three things. 1) Give formal presentations. 2) Produce written reports. 3) Master the art of informal presentations as they happen to occur.
- p214: You must learn to sell your ideas, not by propaganda, but by force of clear presentation. Many scientists thing that their ideas are good and therefore will be widely adopted. This is not the case. Many good ideas had to be rediscovered before they were used - the first time they were invented, they simply didn’t get attention. New ideas are automatically resisted by ‘the establishment’.
- p214: Change does not mean progress, but progress requires change.
- p214: “Good after dinner speeches require three well told jokes. One at the beginning, one in the middle to wake people up, and one at the end so that they remember at least one thing you said!”
- p214: You might think that you don’t have the freedom to work on what you believe would be best. Neither did Hamming. Rather, he had to work for many years to establish a reputation for doing important work (in his own time), so that then he was given the time to do important work.
- p214: Is the effort required for excellent work worth it? The main benefit of the effort is changing yourself - if you end up where you want to be that’s great, but the most important bit is who you are when you get there. You must regularly extend yourself in life, and you need to pick goals worth striving for.
- p215: It is generally easier to succeed than it first seems. There are almost always some opportunities for you to select from - you must develop your style for how to live your life how you want to. Nobody told Hamming most of these things, he figured them out for himself. Now that you have a guide, you have no excuse ;)