My response to Nat Friedman’s set of personal beliefs

The following is my reaction to reading what Nat Friedman believes: http://nat.org/

1. ”Reshaping the universe to our preferences”

What we have done to the earth is a testament to humans prioritising humans’ preferences, at the cost to other living beings and the environment, and as we are learning now, comes around ultimately to us – we are all connected.

Technology could also protect our environment but to have technology carry that as its conscience, that aim must be embedded in our preferences. In our desires, we know very well we are not perfect creatures, responsible protectors, that this conscience must counter our natural tendencies towards selfishness and greed.

Speaking of preferences, I would prefer to differentiate between technology and knowledge, and while we’re at it, wisdom. It is beneficial for us to know the difference.

2. “Energy is a necessary input for progress”

I agree, energy does work (sic), and progress only materialises when something happens and not when nothing happens. Or does it?

But understanding the Chi in every living thing and how to efficiently channel it to worthwhile purposes, I agree with that.

I will tell you a secret of my productivity: when I work, I really work. Full focus and no distractions. When I play I really play 🙂 I cluster my activities and ride the crest of my high energy and fully immerse myself in my low tide, downtime, without guilt or thought for the unfinished things. Understand the different waves of your energy, what are the best activities for them, as well as your circadian rhythm. Eat well, sleep well. Perhaps it’ll work for some.

The long-term aim is to be the master of your own time and not be subject to others for what you want to do and when and for how long. That is one of the true freedoms.

2b. “It’s easier to work on things that are exciting to you”

Sometimes, we have to work on things that bore us because it is our duty, our giri. The reason for being. Not everyone is fortunate to have the option of pursuing what excites them most. Sometimes rewards come in not self-fulfilment but in knowing that because you’re doing something that others are loathe to do or incapable of doing, you have saved the day. There is worth in doing the inferior, the less glamorous, the ‘no accolades’ things, as much as in doing the bigger things.

3. “Do it fast”

The community of sloths might take umbrage at the phrase “slow is fake”. The winner of Fields Medal, June Huh, might be offended that his working only 3 hours a day is too little, too slow.

Too often there is fortuitous serendipity when a project temporary stalls. Reorienting and collecting further information and guidance takes time. Rushing things might result in an inferior outcome, or worse. Taking a pause to consider whether you are moving in the right direction, or to gauge whether the circumstances have changed, matter in the long run. 

More important than speed is constancy in the right direction, that’s what “The Tortoise and the Hare” taught us.

4. “The EMH is a lie”

It is neither wrong nor right but an approximation of one type of market mechanism. To understand the limitations of the EMH is to learn the wisdom of Shannon and information.

5. “Many of the things we believe are wrong”

The way the accumulation of knowledge and understanding work is through fits and starts. We begin with a premise and gather evidence that supports it, and it works, for a time. Until some genius comes along and instigates a paradigm shift and we learn the limitations of our understanding in that particular framework, throw away that tray of tools for a new set and begin again. Has always been thus until we encounter the boundary of our viewfinder of reality and discover that our sight would be clearer, more accurate with a new pair of glasses.

Why has it always been thus? Because humanity has to grow and mature in our capacity for absorbing a new knowledge before we are able to graduate to that more advanced tool for thinking. It will come to us once we are ready. For example, I would explain electrons completely differently to a 10 year old compared to how I would explain it to a physics undergraduate.

6. ““Great” individuals should be allowed to micromanage”

Agree to some extent. Compromise hinders great vision. However.

My father littered our house with management books when I was growing up that I couldn’t help but read them. Edward de Bono is a good source. “How to Make Friends and Influence People” by Dale Carnegie and “Adventures of a Bystander” by Peter Drucker are good too. Old books that withstand the test of time.

“Continuous cooperation” and not a “one-off compulsion” is the key phrase here, unless you like to fire people often. Even then, you lose the cumulative experience and camaraderie of the group. Above all, treat people with respect.

7. “Smaller teams are better”

More important than the size of the team is the composition of the team. 

(By the way, I abhor the labelling of people as “mediocre”. It stunts your personal growth if you limit the things you see of, and learn from, other people because you’ve already decided they are mediocre.)

The composition of a group determines the condition of the discussion. At either end of the spectrum lies “major agreement” and “major disagreement”. In between we have “minor agreement” and “minor disagreement” of varying degrees. 

Let’s imagine a group trying to make a decision for their business venture or a committee on public policy. What could induce the group to immediately agree on a decision? It is either because there is a) so much established information that it’s very easy to concur, or b) there’s very little information known and it’s agreed that that is all the information you can ever obtain at the time of discussion, or c) the group is homogeneously populated that each brings the same set of information to the discussion but the group wrongly surmises that there is no additional information, even though there would be if the composition of the group was more varied. 

The ease to come to a conclusion based on the available information helps a group to come to a quicker agreement. But is it the correct agreement to come to? Situation (c) increases the chances of agreeing on the wrong decision. 

What could promote immediate disagreement in a group on a particular topic? The probability of immediate disagreement rises when the group is diversified, possessing different levels of expertise, different access to data or information, different background and experiences, etc. Immediate disagreement stems from the significant mismatch of what one group member is saying with another member’s circle of competence and personal beliefs. 

The higher the number of mismatched interactions between members, the greater the disagreements within the group. Immediate disagreement is not only caused by the characteristics of the individuals, but also their priorities. E.g., some care about the means while others only the end results. Or care about outcome A more than outcome B. Or some feel that the costs are worth it, and so forth.The problem with a “major disagreement” group is that it’s very difficult and lengthy to get things done as the members of the group find it hard to come to an agreement.

“Minor agreement/minor disagreement” is where I think the sweet spot lies. The composition of the group should be varied enough that the chances of coming to the right decision are maximised, but not so varied that the group comes to a standstill and no decision is made at all. A ‘sweet spot’ perfectly varied group is just an ideal of course, to be aspired to but not truly achievable in real life. Maybe someone should write a book on how to assemble at least a close approximate, that would be useful I think. Or the assembling could be outsourced to a specialist firm. Catalogue all the attributes of potential members and let algorithms recommend a combination. There must be a way to do this. Imagine the improved productivity!

8. Where do I “get my dopamine”?

No idea, to be honest. Different things at different times. I am simple in my pleasures and complex in my needs. “The answer is predictive of your behaviour” – what does my answer then, say about me? 🙂

9. “We are tied down by invisible orthodoxy”

How do you cut down these restraints? By awareness and by voice. Speaking up helps, and if it doesn’t, then it’s time to change your environment, be it a new job, a new place, or others.

I wish I am as optimistic to say that the “laws of physics are the only limit”, but I have encountered enough orthodoxies in my way to know that that isn’t so. In the very, very long run though, I agree with the statement.

10. Conclusion

Approach challenges holistically but don’t forget the feeling of completion, of being in the zone of perfection, when industriously attending to the little details. 

Approach life and its conclusions with a huge, huge dose of humility, remembering the lessons of failed dreams. We are not creatures of ambitions without values, without compassion. 

And finally, as long as you breathe, hope.

@Amni Rusli

The Legacy of Theresa May

The most frustrating thing about Theresa May must be the way she handled questions during interviews and PMQs. It was almost like she was a robot that gave pre-programmed answers regardless of the input.

Phrases such as “deliver on the results of the referendum” or “take back control”- and who can forget “strong and stable” which has its own Wikipedia page.

As a leader May’s mistakes were many. It’s debatable whether the errors arose from the circumstances she was put into. The many impasses she created could well be due to her being, as a male Conservative party grandee described, a “bloody difficult woman”.

This is all forgivable, had the stubbornness been based on being right or having a very good reason, but often times many of us watching felt that she had committed to a position because and just because it was passed down to her by predecessors. She was a car stuck in the first gear and could never transition to the second, third, or even reverse if need be. This lack of nimbleness, this immovability has brought the UK to where it is today – massive economic uncertainty and a political stasis.

Undoubtedly, Theresa May felt very proud to have fallen into the role of the second female Prime Minister, with hopes to even outshine Thatcher’s legacy. Reportedly, when at Oxford, she was not pleased upon discovering Thatcher had become the PM because she herself wanted to be the first.

From the very beginning May had wanted to give the impression of a strong leader. Strong words soon accompanied this aspiration such as “Brexit means Brexit” and she told her cabinet that “politics is not a game”. Regrettably, May continued to adopt this clumsy posturing well into her Brexit negotiations.

On 29^th March 2017, disregarding prudence, absent of due diligence and against expert counsel May rushed a letter to Donald Tusk triggering the article 50 to begin the Brexit process – complete with a veiled threat of withdrawing security cooperation should the EU27 fail to deliver on a trade agreement. Such was the unwise show of strength made by May and her advisors. This kind of thinking permeated her whole leadership. She mistook inflexibility for strength and knee-jerk reactions for decisiveness. In justifying these actions she skirted hard truths and practical realities, trying to convince herself and others that this is “the will of the people”.

Red lines in the negotiations were drawn surprisingly early. In negotiations, there are normally two different types of red lines. One type is the condition you put on the negotiating table to extract as many concessions as you can while the other type is the condition based on existing circumstances you’re unable to change because it’s baked into your structure. The EU’s red lines were made with keeping the integrity of the Single Market in mind. The EU’s red lines were 1. sequencing, 2. the four freedoms covering persons, goods, services and capital and 3. that there is no hard border between Northern Ireland and Ireland.

At the time, it was assumed that the protection of rights of the UK citizens in the EU or respectively the EU citizens in the UK were more or less agreed. It was only a footnoted red line because of the assurances May gave and because the EU team believed that both parties understood it was the right and fair thing to do. Having said that, I’m sure that many still have May’s speech during her party conference ringing in their ears when she said, “But, if you believe you are a citizen of the world, you are a citizen of nowhere. You don’t understand what citizenship means.”

Perhaps May should have asked the EU citizens who lived here in the UK for over twenty years what they think about this statement or looked at the numbers of British people frantically tracing their Irish roots as to apply for an Irish passport and asked them what citizenship really means.

Michel Barnier helpfully drew his now famous staircase diagram, spelling out the freedoms and access for every increase in the number of red lines. Of all the UK’s red lines, ending freedom of movement became a priority borne perhaps from her previous position at the Home Office. Alas, it meant that the upper rungs of the staircase, a “very close and special relationship”, was not available to the UK.

Ending freedom of movement appeased the portion of British citizens who felt threatened by immigration numbers but at the same time angered many in businesses and academia who rely on immigration. We shall see what the long term effects of ending FoM will be on the viability of many business sectors as well as the prestige of British universities.

It was not an economically wise decision as the UK’s comparative advantage is overwhelmingly concentrated in services industries. For the financial sector, the loss of passporting means the loss of the UK’s status as a financial centre, including its ancillary services that are major drivers of UK GDP.

May seemed to be repeating the same act of miscalculating her odds, betting erroneously that her gamble would work and finding time and time again, that after the dust had settled down her gambit had landed her in even deeper waters.

As she became further and further mired within her red lines that should never have been, her statements became more and more canned and robotic. A similar rise of unsubstantiated opinions from questionable academics about which Brexit is best were dispersed to the public on the BBC and other supportive media.

Those who knew better tried to correct the misinformation but without the loudspeaker and active rubberstamping of the government, it was hard to reach the general public. Many still do not understand what they are asked to give up in return for “sovereignty and a proper Brexit”. Those sounding alarm bells are branded elitists who condescend and think that the people didn’t know what they voted for.

May steadfastly surrounded herself with people who would deliver the ultimate “take back control” and banished those who would counsel a balanced approach. Brexit secretaries came and went so frequently that people had trouble keeping track who was responsible for negotiations. Same with parliament – here is a list of cabinet resignations during her premiership.

It was nothing short of a shamble, a painful spectacle to watch as May dismantled UK’s global reputation as a competent, moderate government piece by piece. Her increasingly confusing stands and chaotic parliament opened up a door for stronger populism to spread through the country. A movement brandishing the promise of getting Brexit done, whatever it takes. A recent poll of the European election showed that the Brexit Party is leading. What this means for the future of the UK is frightening.

To her credit, May did eventually concede that her deal is flawed, claiming, “I would say don’t let the search for the perfect become the enemy of the good because the danger there is that we end up with no Brexit at all.” However, did she sufficiently spell out what these flaws are to the British people? Will the failure to educate the public on the issues of Brexit be held against her once she is long gone? Will future generations still be enduring the negative consequences of Brexit?

Although the biggest harm of May’s creation was to embolden the extreme Brexiteers in her party, perhaps the most significant service that she did for the country was to actually keep ‘No Deal’ at bay, despite her previous mantra of “No deal is better than a bad deal”. She could have taken the easy political route and walked away from negotiations and refused the multiple extensions. Deep down inside, she knew that a ‘No Deal’ would bring the UK to its knees and for this, she hedged and hummed and tried to ram “Theresa May’s Deal” not once, twice but three times through parliament. Thus in the end, this bloody difficult woman really did love her country after all.

Pooled Autonomous Vehicles

Fully autonomous vehicles are expected to arrive within the next ten years – billions invested by almost all major car makers and many technology companies will ensure that this is not just wishful thinking. With no driver required the concept of owning a vehicle that sits idle 95% of the time however looks increasingly odd. The London and Silicon Valley based think tank RethinkX therefore makes radical predictions about the future of mobility : 90% of all miles driven will be autonomous and shared, as early as 2030. The assumptions in their research are worth studying in greater detail in the future but they appear mostly plausible and interested readers will find plenty of articles and videos that expand on this. In this post I’d like to make a few contributions of my own that focus on pooling and vehicle design in particular, which could make this future even more appealing and affordable.

Benefits of AVs in general

There are so many benefits to autonomous transport as a service, so here is just a quick rundown without going into too much detail:

• Lower cost, freeing up resources for other spending
• Freed up space for parking
• Lives saved from far fewer accidents
• Longer distance commutes possible, enabling more space for housing and people
• Empowerment of those without transport, including children and elderly
• More physical connectedness of people
• Increased productivity in transport as people work or sleep
• Parents freed up from transporting children
• Transport speeds up as road congestion decreases
• Compared to public transport wait times at interchanges are eliminated
• Drivers freed up to do more interesting tasks

Given these many advantages it is highly likely that the public will soon prefer transport as a service outside the most rural areas. This is because autonomous vehicles (AVs) will be available more cheaply and more reliably than individually owned vehicles and even some forms of public transport. It is interesting that Elon Musk has explicitly omitted buses in his updated master plan, instead believing that AVs will take their place at similar or better per mile cost.

Pooling

One key remaining question is whether this transport as a service will be mostly pooled or not. There are two arguments in favour of a largely one party/one vehicle model:

1. Privacy and comfort: people are used to privacy in their own cars, which is often the reason they currently prefer driving over public transport. Sharing a confined car space with strangers is not very appealing compared to stretching out in your personal space.
2. Predictability and speed: no detours to pick up or drop off other passengers and no extra wait to be find other riders

By contrast the two main reasons in favour of a more pooled model are:

1. Cost: the marginal cost of transporting another paying passenger if they share the same pick-up and drop-off points are close to zero
2. Congestion: road space is limited, especially at peak times and speed could be increased in aggregate if fewer vehicles were on the road

There is a way that these two advantages of unpooled vehicles can be fitted into a pooled solution that satisfies all objectives. Privacy can be created with a new vehicle design while speed comes down to a software solution.

Vehicle design

Based on a large minivan design of approximately 5-6m in length an improved pooling friendly design would look as follows:

1. The entire length of the vehicle would be taken up by the cabin, similar to buses. The shape may come to resemble a slightly larger version of the VW microbus and its latest prototype. There is no need for an engine compartment at the front, all batteries and engines can be fitted into the floor. Internal combustion engines won’t be competitive by the mid-2020s according to multiple forecasts (UBS and others). As AVs are far safer the need for a front crumple zone (or even a Tesla frunk) disappears. The raised height of the floor also allows less incursion of the wheel arches into the passenger space. The result is a long, almost flat floor plan with a mostly rectangular passenger cabin on top that allows for new interior design ideas. Poor aerodynamics from the boxy shape are a lesser consideration given increasingly cheap energy – like a bus or RV maximizing interior space for a given vehicle size is the most important feature.
2. To address the privacy and comfort the cabin would be divided into 2×2 compartments, each with its own sliding door. The compartments would be separated by half-height dividers that are raised to full height by default, similar to the sound-proof dividers in current stretch limousines between the driver and passenger compartment. A request from both sides would be needed to lower them for groups travelling together. The windows could similarly be covered, frosted or tinted to enable total privacy.
3. To further enhance comfort each compartment would contain a seat similar to a fold-flat first class airline seat. This would allow passengers to sit, recline or even lie flat to sleep. It is likely that seat-belt requirements will be removed as AVs will be treated like buses or taxis. Similar to such airline seats there will be fold-out tables and screens that can be lowered from the roof, in case passengers want a surround view of the car or get entertainment from their devices or in-car services onto a bigger screen.
4. Given the length of over 2m in each compartment there is room to use the space flexibly. A jump seat at the front of the compartment could be included to be used by a second person for shorter trips, similar to the rear-facing jump seats in London taxis. As there is no boot, luggage would be stored in the large compartment itself, either behind the seat or at the front. If more space is required an additional compartment can be booked.

The interior design would address comfort and privacy, despite sharing the overall vehicle. The best analogy here is a first class airline cabin, with the added privacy of partition walls. None of the pictures in the slideshow below quite hit the mark but there is a combination of all of them could be mashed up into a final design.

Scheduling and software

To address the second concern of predictability and speed the solution lies in software and network effects. A passenger arranging transport between two points would book by compartment, not people, depending on the size of the party and luggage requirements. In most urban environments it is likely that there will be many similar trips booked within a short time of each other. In order to achieve the benefits of pooling such trips a miniature hub-and-spoke model would be adopted. The passenger would most likely set off alone in a vehicle until reaching an optimally located spot at which up to three additional feeders would arrive at almost the same time. Passengers would then change to the vehicle that would take them into the area of their shared destinations, before repeating the changeover again near the final destination. No detours for pool passengers! If carefully timed, which is very possible given increasingly sophisticated software and large number of users, such changeovers should not add more than a few minutes to travel time. This in aggregate would be repaid by much fewer vehicles on the road that would allow higher speed. Cities may indeed mandate this.

This scheduling system is in effect a highly personalized form of public transport, gathering traffic along trunk routes, lowering congestion, but at the same time enabling true door-to-door convenience. Network effects would enable the provider having the most vehicles on the road to provide the tightest schedules and lowest costs, creating a market that will most likely become oligopolistic, something that may in time attract the attention of regulators.

As an aside, the traditional way of hailing a taxi may actually make a comeback, e.g. in situations where someone’s phone may not work. Exterior cameras on AVs on the road could pick up a particular hand motion of a person seeking transport, which would then activate facial recognition and acknowledge the request on an exterior display. The service would then send a nearby vehicle to pick up the passenger, who would input the destination on the on-board systems. As this is much less predictable than pre-booked traffic this would be a service with a significantly higher fee.

It should also be mentioned that AVs can drive in tightly packed convoys that greatly increase the number of vehicles that can travel on a given stretch of road at a time. Traffic lights could be prioritized for such highly utilised and tightly packed vehicles to encourage adoption of this mode of transport. Mass rapid transport will likely shrink greatly outside the densest urban environments.

Cost implications

The RethinkX report calculates a very low per mile cost of $0.08-$0.24, which dramatically undercuts all other forms of transport. This translates into an annual cost for a daily 30 mile commute of just $550-$1700, based on 230 working days. However, there are two omissions in these figures: peak demand and deadheading (i.e. non-revenue miles). Peak demand is reflected in Uber surge pricing, a widely disliked but necessary increase in prices of up to 4x to bring supply and demand into equilibrium. Underlying the low cost calculations made in the report are high utilisation rates for all vehicles. To avoid surge pricing without pooling a much larger fleet would be required, which would sit empty much of the day, increasing per mile costs for all. By relying more on pooling during peak hours prices can be kept low and predictable throughout. Conveniently this is also the time where matching riders is easiest. Deadheading is also reduced in a hub-and-spoke model as only the return traffic on the trunk routes may be empty while the feeders could constantly be kept full, as the hubs are themselves moving constantly so could be organized such that empty local trips are minimised.

The result is that pooling may not reduce prices much further as indicated by the report but instead could keep them constant even at peak times.

Long distance transport

Another area that will open up with such pooled AVs is long distance transport. Door-to-door travel, possibly overnight, to a destination a few hundred miles away is suddenly far more comfortable than the alternatives of trains or planes with long interchanges, pre-boarding procedures and transport to and from the hubs. As such trips are most likely planned well in advance it is quite possible to create a small pool of passengers, as only a maximum of four is needed. As this traffic would mostly occur at off-peak hours without much deadheading this should enable even lower per mile cost than local traffic. Given greatly increased safety and zero emission of AVs there should also be no reason to maintain speed limits at the low current levels, allowing for even greater distances. There may be a small subset of the vehicle pool that has larger battery capacities for the longest distances but during the day they’d act like all others.

This form of transport could become the most convenient for anything up to a few hundred miles, beyond which flying is usually the fastest. This may squeeze out high-speed rail in many situations.

Delivery

To drive even higher utilisation these vehicles could serve as delivery vehicles during off-peak hours. After establishing that the recipient is available they would walk up to the car and after some interaction with their phone the door would open. To ensure only their package would be taken a number of steps could be taken:

• Large, bulky or valuable items could be transported in their own compartment.
• Multiple, lower value packages could be put in a single compartment. Cameras would be switched on to both guide the recipient to the correct package (augmented reality) and to ensure only the correct package is taken. Violators could be instantly reported.
• Alternatively, locked pigeonholes could be installed for delivery hours in one or more compartments. The installation with pre-filled pigeonholes could be done at delivery centers within minutes on top of the lie-flat seats. The recipient would then only get access to their pigeonhole.

This is probably inferior to other ideas laid out in another article describing a van and drone combination   but with near zero marginal cost at off-peak hours this could be a complement to other delivery services. The ability to deliver larger items in the huge luggage space in particular would be a complement in situations where drones would not work. A service similar to Uber Eats could also be viable for the very short local distances required there.

Using these vehicles as both passenger and delivery vehicles, maybe even at the same time will again cut costs for all services and increase utilisation.

Ethics and other uses

In the context of AVs many ethics questions are being raised, such as the behaviour in accident situations, external cameras recording streets and faces. I’d like to address two specific questions here:

1. Privacy: should cameras be installed in each compartment? Preferably, cameras should only be activated in a few situations: between riders to inspect the state of the cabin to determine whether cleaning service is required, during package delivery or for children travelling on their own so parents can watch over them. Parents not having to do school runs and taxi service to friends and activities will be a major productivity driver!
2. Does society accept if such highly private spaces on the road are used for activities not primarily related to transport? In many cases we’re fine with that, e.g. food trucks, mobile clinics but what about using a car for taking naps or even as mobile hook-up sites? What may be happening rarely in the few stretch limousines around may become more common in ubiquitous AVs. A free society should judge whether these cases impinge on the freedom of others so we should normally have no business in restricting what people are doing in these new personal spaces.

Conclusion

Given very compelling economics it is very likely that the predicted comprehensive transition from individual ownership to transport as a service will materialise in the near future, whether by 2030 or slightly later. In the inevitable debate of personal mobility vs. increased congestion the partly pooled model outlined in this article is likely to satisfy all concerned. In addition, the proposed single, unified multi-purpose vehicle design may lead to the highest utilisation and lowest cost outcomes. City planners will be pleased that despite the reduction in mass public transport the number of vehicles on the road will remain manageable due to high occupancy. Passengers will appreciate the very low cost, yet first class experience while maintaining personal transport speed, flexibility and privacy.

This slideshow requires JavaScript.

House Prices One Hundred Years from Now

I would like to commend Professor David Miles for a fascinating talk on the causes of rising house prices, going beyond the usual suspects and looking at technology and incentives. Given the short lecture format not everything could be covered, so I would like to add some of my own thoughts from the point of view of economics and technology.

Finance

The decline in interest rates since the early 1980s, both nominal and real, have certainly had a significant effect on house prices. The decline was driven by disinflation and the resulting lower volatility in inflation, which allowed real rates to go much lower. A global savings glut and more recently, monetary stimulus are additional factors. While this is a well-known fact, it is nevertheless important in any discussion about house price growth.

These factors raised affordability at a fixed share of earnings. As much as economists like to look at everything in real terms, nobody in the 1970s could borrow against future inflation if monthly payments would be crushing at double digit interest rates. The longer duration made possible by lower rates would therefore unlock earnings decades in the future to borrow against, rather than being heavily front-loaded while interest rates were high.

Banks went along and raised their limits for maximum lending to earnings, with a significant lag, partly because it takes a long time for stress test interest rates to reduce as well. In other words, when banks ask the question of whether can you afford your repayments if interest rates rise to x%, the x is decreasing very slowly.

However, at the current zero lower bound, there is not much upside to house prices from this factor alone but plenty of potential downside.

Affordability

What is often missing from the analysis of house price growth (without undertaking a literature review) is a breakdown of how people are actually financing house purchases far above the maximum bank multiples of 4x earnings. Detailing the sources of this further equity or debt would help to determine whether any of these are sustainable.

Increasing equity via repayment of previous homes certainly is sustainable, inheritance and parental gifts probably is too (unless taxed too heavily). On the other hand, rising home equity from appreciating prices may not be counted on as a source of increasing equity as people usually move to more expensive houses than they were coming from. As pointed out in the lecture, older people moving from expensive to cheap accommodation are becoming a rarer feature of the housing market.

Another route to raising affordability is raising the maximum multiples of mortgage to earnings and reducing minimum equity requirements. Having just been through a harrowing financial crisis this is unlikely to occur for at least a decade.

Consumer preferences

The lecture covered the topic of trading more space for other things. It appears that the appetite for more space is unbroken, doubling in the last few decades in the US, which are a truer expression of preferences than the artificially constrained picture in the UK. There is a possibility that ‘peak stuff’ may put an end to this trend. ‘Software eating the world’ has removed the need for many devices, disks and books in the home, the switch to LCD TVs hanging on walls from bulky tubes has reduced removed many TV stands. VR could one day even make these obsolete and with it the distance between screen and seating.

A trend towards eating out more or relying on ready-made foods is diminishing the role of the kitchen, which could shrink in decades to come. Or it could absorb the role of the dining room, which appears to be on its way out. Storage space for groceries could also shrink as people switch to more just-in-time delivery that’s getting cheaper all the time rather than having to store months of supply from buying Costco sized grocery lots. The sharing economy could reduce the need for much stuff further, e.g. power tools, lawnmowers and other rarely used items, where lower transaction costs make it much easier to hire the required tools or skills by the hour. Of course, people have been creative in filling up the space with other things, e.g. home gyms, games rooms and ever larger bathrooms with saunas, Jacuzzis etc., as I have discussed in this previous post.

One piece in the growth of demand is however certain: the sharing of space of unrelated people (or of people in unhappy marriages) or living with parents in their 20s and even 30s due to cost would end if space was cheap enough.

Building technology

The example of very tall and slim skyscrapers in the lecture is only one of many new technologies, maybe not even the most significant. Prefabrication and 3D printing could have a profound impact on the time and cost of construction. In 2015 a 57 story building was put up in 19 days and there are now early prototypes of 3D house printers and automated brick layers. Building and outfitting have not really improved in their productivity for decades. Assembling modules off site or autonomously with robots/3D printers instead could lead to large cost decreases.

By reducing the cost of going up in height it may be possible to increase the incentive paid to landowners with existing structures to tip the balance in favor of the agreeing to higher plot ratios, as discussed in the lecture. Similarly, not all countries have basements by default in most houses, with cheaper digging technology (unlike other building tech this is currently not heavily researched) this most unobtrusive kind of extension of living space would be of great benefit.

Planning rules

Focusing on the UK at first I would like to reference a number of publications by Policy Exchange, starting with this article. It lays out very clearly that the lack of local incentives is one of the main reason behind the lack of suitable space to build on. Fixing this would unlock a great amount of land that could lead to better accommodation for a great many people while remaining affordable.

Giving locals a share in the upside from new residents rather than just the downside of more crowding is something a more federated system of government could deliver, see Switzerland as an example of this. In addition, the oligopolistic hold on the housing market and land banks enjoyed by the large developers could be broken by selling land to individuals who would build on it instantly, rather than holding out for higher margins later. Examples such as Austria where 80% of properties are self-built support this.

I’d like to point out in particular that the so-called green belt around London in fact includes a great deal of farmland within a few miles of fast train connections to the center. This is a colossal waste of resources. Parks and woodlands are greatly valued by local residents and in frequent use and should remain protected. Densification by converting people’s gardens into more houses is usually hated and not a way forward. And while brownfield development is always cited as the way out of this crisis there’s only so much of it and it’s not usually where people want to live.

Surveys consistently show that the most preferred accommodation is a (semi-)detached house with a garden, yet very little of this that’s affordable is being created. Farmland is of low value (in 2010 a reclassification led to a 200x gain), has no use to anyone other than the farmer and in fact is net negative for biodiversity and local health due to spraying of pesticides and fertilizer. Not even the views over the empty land are that attractive. The correct test for keeping land as protected greenbelt should be whether it is accessible and used by the public. Farmland within and just outside the M25 does not meet this test and should be opened for widespread development, especially where it is within short a short distance of fast trains.

Two international comparisons stand out as interesting. In Tokyo it is possible to buy detached houses of decent size within 20 minutes commuting distance of central business districts at $300-400k. This is partly due to excellent transport but also because plot sizes are very small and going up is done in a way that is fairly attractive and preserves privacy. Houses are often torn down after 30-40 years. This may result in slightly higher total cost of ownership but it also enables a much better matching of housing preferences to housing stock.

If cheaper technology enables going from 3 to 4 or 5 floors at similar costs this can be done on existing land as part of the normal replacement cycle, all without requiring additional land and new transport infrastructure. It should also be noted that there are much fewer limits on design of such houses, leading to a much greater variety than can be found in other countries’ monoculture. While it is second best to a house with a garden it maximizes floor space while minimizing commuting time, a trade-off many in the UK would be happy to make.

On the other end of the spectrum is Houston, a city that has not made any attempts to rein in sprawl or impose many zoning rules. The result is affordable housing with plenty of space for a great majority of people. It comes at the cost of longer commutes but as can be seen from the next section technology can be of great help here.

Transport and communications technology

If we are looking one hundred years into the future, it is a near certainty that we will be in an age of autonomous vehicles. This leads to two profound implications:

1. Space currently devoted to parking where people live will shrink dramatically as car ownership moves to a shared model (see recent Economist cover story), creating more land for living, and
2. The radius of where people may travel from increases greatly, because of greater speed and comfort.

In terms of parking, space would free up both in the city centers, around transport hubs and on driveways as a far smaller shared fleet is needed for the same level of mobility than in a model of individual ownership. In residential areas, garages and driveways could be rearranged for more living space or units. Parking structures could be replaced with new high rise apartment blocks.

Travel will take two distinct modes, either home to transport hub or home to final destination, based on the density of the city. The first case usually covers cities in Europe and Asia where the dense urban core already has good public transportation: here it still makes sense to rely on trains to bring people to the city center, however the trains bringing them there could be sped up to much higher express levels with far fewer stops if the trip from house to station could be made autonomous. At the transport hub no parking would be required and the time in the vehicle could be spent sleeping or working.

In sprawling cities, transport will be fully door-to-door. Autonomy there would lead to much better traffic coordination, smaller gaps between cars and shorter journey times. Speed limits could be raised significantly. All this will happen without wildcards like Hyperloop or Boring company tunnels, which could improve on this further.

With regard to travel radius, there are four distances to consider when people make choices about a location to live, in no particular order:

1. commute for employment,
2. children traveling to schools,
3. local shopping and entertainment amenities,
4. distance to community and relatives.

In the context of employment, the rise of remote working is an important factor. While early projections of the death of the office have not played out, a mixed model may be emerging whereby people only commute on some of the days or for shorter workdays, completing a significant amount of work at home. This could reduce aggregate time in transport, so a daily one way 45 minute commute could be replaced with a 3 day a week 75 minute commute, or the commute would be scheduled for off-peak hours to avoid traffic with the rest of the work being done at home.

Much local traffic is about sending children to schools. In many countries parents have a preference for them to be within walking distance. With autonomous vehicles this is a much lesser consideration, children could commute from home to even distant schools in autonomous vehicles without their parents’ involvement, opening up additional employment opportunities to them. This would enable either families or schools to spread out much further than currently possible.

Shopping is being automated at a very fast rate, including groceries. Autonomous delivery is another near certainty over long enough horizons (see my previous post here), cutting down on the number of mandatory trips. While not everyone will adopt such a lifestyle, a significant proportion of the population that’s not interested in shopping as an experience or much entertainment outside the home (e.g. eating out) is enough to reduce demand for living close to such amenities.

This leaves distance to community and relatives as a key consideration. Again, the generation growing up now is far more used to interacting on social media and less reliant on in-person interactions so the importance of location in this respect may decline too. Opposite considerations can come up here as well, whereby it may be as much about avoiding particular communities than it is about finding them.

With communication and transport technology solving most of the distance issues, the main remaining reason for a choice of location is related to status and some unspecified “feel”, whereby city dwellers like the noise, buzz and freedom of a large city whereas more rural residents prefer peace, quiet and fresh air.

Population growth

The model of ever rising house prices rests on the assumption of growing populations, but in many countries there’s not much rural population left to move into cities, birth rates are below replacement level in many countries and immigration isn’t at the scale to offset this and may never be. Life extension technology is at its infancy but over one hundred years could achieve significant breakthroughs, leading to population increases from lower death rates. Absent that scenario, the megacities are the most likely centers of growth, taking population from smaller cities as life will focus more and more around these megacities. The impact on house prices in faraway places then hangs on the question whether truly distant commutes like the one mentioned in the lecture from Somerset to London become viable.

Conclusion

My conclusion regarding the question of 100 year house price growth is as uncertain as the lecture’s. It depends on how many of these threads play out. The finance industry won’t be a positive factor in raising long term prices, at least in the near term, but large scale inheritance could be. The necessity of living near work, school, family etc. will certainly reduce but this could be replaced by a much stronger preference for city living, continuing a very long trend. Centuries-old building technology is due for a much needed shake-up like all other technologies but greater density may clash with preferences. Whether demand for space will keep growing also depend on how space-shrinking technology offsets our natural desire for ever more of everything. Demographic trends 100 years into the future are also uncertain, central predictions are set for declines in most developed countries but immigration and life extension could also change these.

It is only due to human ingenuity that we have achieved incredible living standards, constantly managing to achieve more output with less input. It would be a shame if we couldn’t bend the cost curve for housing as we have for almost everything else in this world. We owe it to future generations.

The Problem With Smart People

The problem with smart people is that they are able to justify their wrong actions with better arguments. This gives them a troublingly unfair advantage over the rest. With that above average ability they attempt to persuade us with explosive fireworks of reasoning that deafen our hearing with their loudness, and daze our sight with their brilliant flashes.

Some amongst us would be easily convinced, while some others, even though we do not whole-heartedly agree, couldn’t find in our minds a path of well laid-out reasons why we shouldn’t just succumb. In this way, those in power have always employed the smarter ones to be marketers of their ideas so that they could increase their already large influence even further.

What then is the defence of ordinary people who have not read Plato nor know Newton’s Laws? Are we defenceless from our lack of scholarship and charming wit?

No, we are not.

In everyone, despite our differences in background and education, I believe there is a sense of what is right and what is wrong, possessing the ability to identify what is unfair, unjust or outright unconscionable. An intuition fully ingrained that does not require a brilliant mind to justify a particular rationality, but rather a compassionate heart to recognise.

Perhaps this is the way that nature balances out those with superior intelligence but no heart, by installing the intuitive wisdom of the ages deep inside those with not as equally brilliant a mind but plenty of heart.

Caveat: I don’t propose a purely irrational, emotional-dependent view either. Instead, a healthy balance of both reason and wisdom-led compassion.

New Possibilities in Autonomous Delivery

A few days ago it emerged that Amazon had put a dozen employees to work to investigate self driving technology. This is a fairly small scale effort that has not gathered much attention but it could turn out to be equally significant as the much more prominent drone effort.

The public is enthralled and at the same time terrified by a future of drones ferrying goods and people around cities. It is hard to imagine the airspace filling up with drones that significantly replace the tens and hundreds of thousands of journeys of people travelling to work and delivery drivers making their drops. While only able to operate in two dimensions the road system remains the best option here as the space around each vehicle can be very small, especially if the majority is self-driving and is communicating with each other to avoid collisions. By contrast even a small fraction of failing aircraft would wreak much greater havoc. The noise coming from even the quietest electric drones would be deafening if there are thousands. And finally the push-back against privacy invasions from drones is not to be underestimated.

This leaves ground transport as the only viable large scale transport option. However, the last yards, as opposed to the last mile, are still difficult to cover for delivery traffic. This is where a combination of autonomous delivery trucks and drones could be very promising.

The basic idea would be that a delivery truck could carry a small fleet of drones that are ready to dispatch packages to the intended recipient. After approaching the destination, a drone would emerge from the roof of the van and fly to the destination.

In the suburbs this could be an area at the back of someone’s house (as in the early Amazon drone videos), or a drop box at the front that only opens for an approaching drone. It could even hover in front to allow someone to take the package. As the back is typically secure it could drop things without the owner being there. The privacy implications would be minimal as the package would only fly a few yards, possibly over your house in order to deliver what you ordered.

In more urban areas you could imagine multi-apartment buildings (without manned receptions) to be upgraded. Similar to individual air conditioning units drop boxes could be installed on every apartment for a few hundred dollars that would open to receive deliveries. If the windows are big enough you could even imagine them to open automatically to allow for drops and the drones depositing the package right inside your home. Balconies are another great potential drop point. Apartments with only backyard facing windows are no obstacle as the drone would just go over the building to reach the other side.

Large apartment blocks without opening windows or balconies and businesses would probably continue with manned desks for accepting deliveries for a while but automating these, including drone flight indoors are possible in a more distant future.

This leaves only very rural areas where we’d see long distance drone flight as there would not be enough packages to require a van to send there. It also opens up otherwise inaccessible places, very important in the developing world.

It could even eliminate the need to park. Instead a delivery van could circle a block, release one of its dozen drones whenever it is closest to the destination building to make drops. It would then return a few minutes later to collect them again, potentially without ever stopping as drones fly out and land while the vehicle is moving at normal speeds. In each case the flying distance would be only a few tens of yards horizontally and a few tens of yards vertically, if they have to go over a building to reach the other side.

There are added advantages. The battery technology to cover a few hundred yards is very unchallenging, unlike the multi-mile journeys from depot to house. The drones could recharge inside the van until the next drop. Given the lower range they could also carry larger weights, making autonomous grocery deliveries including drinks practical. There are fewer safety issues: they’d be close to the ground while making the horizontal journey, avoiding pedestrians and other obstacles carefully. They’d then find a safe square meter or so without people beneath them to make the vertical climb to reach the destination and make their drop, before returning again.

By contrast, a much hyped competing technology of small autonomous ground crawling delivery vehicles seems less promising. There is no mechanism to drop the contents if the owner isn’t there and even small curbs seem like a major obstacle. Drones could hover or even set down temporarily in order to avoid all pedestrians much more easily, while always being able to fly over or across obstacles. For added safety drones could be prevented from flying where pedestrian density is above a certain level. It’s also to be determined whether it would be safest to fly them at eye level (with mesh around the rotors to protect pedestrians) or at 3m height just above people, while still avoiding flying directly above them at all times. Also, the maximum weight of any single drone package could be limited to no more than a few kg. All these measures would result in very safe operation where falling drones should never cause much injury.

Eventually you could imagine all kinds of delivery being handled this way, not just packages but also daily mail, groceries and take-out food. The drone-releasing vehicles could come in all sizes, from scooter sized single drone ones serving time critical take-out deliveries that can swerve through traffic to very large vans that serve large areas.

Another benefit could be to switch away from cardboard packaging, which requires recycling and more garbage truck trips. Instead, you could imagine reusable plastic crates being used. They don’t need to be locked as the grip of the drone would act as a lock until delivery is complete. After announcing its impending arrival a few minutes earlier it may be that a drone landing in someone’s backyard, window or front room would then give 1-2 minutes to the user to empty the crate to take it away immediately again. If that is not an option it could be left in the same spot for pick-up during the next delivery and a deposit system could be used to incentivise their re-use. Closed plastic crates are also better in any weather than cardboard that can get wet.

Completely autonomous deliveries could also be shifted to off-peak hours, including night-times when traffic is very light so this could also help reduce rather than increase traffic.

Unfreedom of Speech

Those who are the masters of platforms have to be extremely cautious in upholding freedom of speech. In an environment where fake news spread with ease but where the education system has failed to produce enough critical thinkers, we need to guard against abhorrent views that leave stains like curry on a cotton shirt. Allowing free speech is not the same as permitting carelessness that normalises hatred, leaving it permanent in the fabric of our society.

How can we tell whether a view is abhorrent or not? One giveaway is that it fosters the feeling of superiority in one group of people above another group. Opinions that are conveyed within a mere hour can justify resentments in those that had already held them. This can undo decades of work that aims to raise the status of minorities, on the basis of justice and equality, overcoming many roadblocks to achieve the same opportunities and recognition.

The counter argument is usually, why not fight falsehoods with truth and facts, or are we scared of a little debate? To that I say, where did that thinking get us with Trump? Can we really assume that everyone will know the truth when presented with it? And let’s not forget those who know the truth but willingly turn their backs on it.

After the Brexit referendum result was announced, 289 cases of hate crime were reported the day after. Again, that’s a record of 289 cases in only one day. This tells us that for some, all they needed was (what they perceived) as tacit agreement from society that their hatred is justified and therefore, can be expressed openly.

Often, the consequences of hateful speeches do not end in the hall where they’re held. The effects ripple out onto the streets and manifest in aggression that may harm others. Some people who come to these events may not only come to listen to the talk, but also to tally those attending, how many there are in agreement with such views. A large enough crowd will make them feel more assured to voice or act out the hostility that has been kept hidden for so long.

My friend in Westminster who works in higher education said that there has been a drastic increase in violence towards minorities in universities. Last year, shortly after Brexit, the police reported that there had been a 42% spike in hate crime and numerous reports of ethnic minorities and immigrants being targeted for racial abuse.

In Croydon two Fridays ago, a 17 year old Kurdish boy was severely beaten by a mob of up to 30 people when he admitted to them that he is an asylum seeker, witnessed by onlookers from a nearby bus stop. He was left with a fractured spine, shattered eye and bleeding in the brain. He is now fighting for his life.

I too, have been mistreated (though not beaten, but verbally abused) at an art gallery by those brazened by Brexit, who thought that I did not belong there and that now it’s socially acceptable for them to say so. And that wasn’t the only time. Now I am fearful of going to places where people might think that I don’t belong. And when I do, always keeping in mind to be out of the way and become a wallflower. This has made me think that freedom of speech resulting in the loss of freedom for others is not really standing up for freedom at all.

Masters of platforms need to exercise some wisdom and common sense. Don’t close your eyes to the fact that by permitting or encouraging, you are complicit in fostering hatred and division. Everything has its good and bad. Freedom of speech too has its evil, perhaps not to the listeners of the speech, but rather to the group that can get hurt by the actions resulting from the speech’s message. Holding up pure ideals without taking into account human behaviour and eccentricities, their capacity to overreact and be riled up, is being negligent in the protection of the fragile groups of society.

When they are young, by all means, teach them the sacred value of freedom of speech. But when they are a little bit older, do teach them about moderation and practicality of holding onto ideals, including their unintended consequences.

I believe in free speech. I believe, I truly believe. But falsehoods and hatred can win if we are not careful.

That’s what I’m afraid of.

 

But wait a minute

Probability of The Unforeseen

Lord Mervyn King, the former governor of the Bank of England was once asked about the low interest rates and whether they were good or bad. He answered that they were good, because that meant young people can take out mortgages to buy houses. “But wait a minute”, was that what really happened? Not quite. As a result of the low interest rates non-young people started buying to rent and this drove the house prices even higher, putting the house market out of reach for the young people who were trying to get onto the housing ladder.

Was this an oversight by Lord King? Perhaps. It is hard to predict the future, but more often than not there are unforeseen consequences. Although we could not know beforehand what these might be, we should always assume the probability of the unforeseen happening to be not zero.

What is the usefulness of this assumption? It’s not as if we could prepare for the unforeseen or as the phrase often attributed to Donald Rumsfeld, “unknown unknowns”. Instead, by always having this at the back of our minds, hopefully our decisions are imbued with greater prudence and diligence, aware that the outcome of our decisions may materialise within a range and not in an accurate, specific bulls-eye way as we often wish it would.

Local versus Global

In a brief essay, Marti Leimbach writes about her hard life, arguing that privilege does not come automatically just by being born white. Despite sympathising with her I thought, “but wait a minute”, when making a case shouldn’t we first differentiate whether the points she makes are local or global?

Whereas her situation was due to bad luck and localised to her person, bad luck that could have fallen on anyone, the negative effects of racism (and yes, that includes lack of privilege), sexism and all other discriminatory ‘isms’ do apply universally based on colour, gender, sexual orientation or class independent of personal situations and luck. To conflate your own personal situation with society-wide challenges does not advance the discussion on the definition and exclusivity of ‘privilege’. People often try to pick out a single abnormality to disprove a whole case, especially those who write to distract the readers from the real issue.

Even in mathematics a distinction is made when describing local and global solutions. Every additional constraint which might appear as the problem demands, would require the narrowing down of the set to one or a few specific solutions of the formula, away from the global optimum. On the other hand, if you start from the vantage point of a local optimum, you may wrongly extrapolate that this is the global solution too.

Proper Sample Size

“But wait a minute” thinking may also help to prevent us from jumping to conclusions. Let’s say that a person attends an interview equipped with high recommendations from previous employers and a nearly flawless record performance of many years. The interviewer for some reason or another then summarily dismisses the person based on this single interview. Is this outcome correct?  Can suitability for a job be determined based on one interview?

Alternatively, if someone is recruiting an athlete and dismisses him as a candidate based on a single field performance, we would say “but wait a minute” that’s ridiculous, that’s not enough observation to know whether he is a good athlete or not. Some would even say that this is not fair, we have to see more of him on the field. It could be that that day he was ill or still recovering from an injury.

To come to the right conclusion and therefore outcome, we need to have a proper sample size suited to the situation. Here, I’m reminded as well of a lecturer at the MIT who acknowledges this by allowing his students to take the better marks of the two major exams, saying that “Everyone has a bad day!”.

In case you wonder, Google, who is well known for measuring everything, found from their research that the marginal benefit of an additional job interview diminishes after the fourth, so maybe there is value to making the intangibles measurable after all.

At this point you might think “but wait a minute” is just a disguise for adopting good mathematical practice in your thinking, and indeed you may be right. Leonardo Da Vinci said, “No human investigation can be called true science without passing through mathematical tests; and if you say that the sciences which begin and end in the mind contain truth, this cannot be conceded, and must be denied for many reasons.”

‘But wait a minute’ thinking as the phrase implies, is about taking a pause after we have come to a conclusion and questioning whether it was the right one. This one minute of self-check is perhaps sixty seconds longer than most people would ever give themselves the luxury to ponder.

Learning and Go

I must admit I didn’t take much notice of the DeepMind’s self-challenge to defeat a human Go player before. Today, their algorithm AlphaGo beat Lee Sedol for the second time, and that made me wonder how they did it. According to Nature, “the average 150-move game contains more possible board configurations — 10¹⁷⁰ — than there are atoms in the Universe, so it can’t be solved by algorithms that search exhaustively for the best move”.

From the same Nature article,

AlphaGo plays in a human way, says Fan. “If no one told me, maybe I would think the player was a little strange, but a very strong player, a real person.” The program seems to have developed a conservative (rather than aggressive) style, adds Toby Manning, a lifelong Go player who refereed the match.

Do you know how a magic trick gives wonder? Well, just like if you knew how a magic trick is done, studying machine learning takes the marvel out of the human-machine Go game duel. On the other hand, understanding machine learning makes you admire the achievement even more, knowing that one, some of the very best minds must have worked on the algorithm, two, the real ‘wonder’ application is ultimately not the Go game, and three, believing that machine learning is to data what Jethro Tull was to the agricultural revolution.

According to Wikipedia, Jethro Tull “perfected the horse-drawn seed drill in 1701 that economically sowed the seeds in neat rows. He later developed a horse-drawn hoe. Tull’s methods were adopted by many great land owners and helped to provide the basis for modern agriculture”. If you think about it, it’s not so very different. For machine learning, the data has to be prepped first into neat little rows and columns and later ‘harvested’ for ideas.

Pondering it, Go naturally lends itself to the classification problem, more specifically, the tree search and the boosting method. Now game tree is not a new thing, it has been used for solving chess and other arcade games such as RushHour, the car parking game app my sister is so great at solving. According to this write-up, minimax tree search can calculate the high probability of the opponent winning and abandoning that node of play, otherwise known as alpha-beta pruning. However, with some cases, “it is simply too expensive to exhaustively search the game tree. The search tree grows approximately as M^N, where M is the number of moves available per turn, and N is the number of total turns in a game”.

Thus, more than brute force is required. This is where the old (but back in fashion) deep neural networks idea comes in. What we do is build a ‘matrix of positions’ and another of the ‘probability of winning’. Mapping the positions onto the probability of winning gives a guide as to which next move is the best through a simple binary classification, without a lookahead. This is known as value networks.

My suspicion is on top of that, they classify local areas and map that also to the probability of winning. Doing this would speed up the runtime – but this is just my guess. Is it then about mapping? No, it’s a search problem, and search is not about maps. Search is about making choices when you explore the tree. This is known as policy networks. Hence, AlphaGo is a hybrid of two networks.

For those who wish to understand this in depth, do refer to this. I’ve blogged before on gradient descent here. On the search algorithm, here. From their paper published in Nature, “Mastering the game of Go with deep neural networks and tree search”,

Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play.

That, in a nutshell, is how it is done.

To sum, what did we use to do with data? We used to do the thinking first, model it and apply the thinking to the data. With machine learning, we let the computer learn based on the data what the best idea is for it to achieve the goal that we have set out. With one subtle shift of this responsibility from human to machine, many things become possible, including playing Go.

There is still some apprehension in many places about these techniques, often claiming over-fitting. In fact, machine learning is much more transparent about this problem, using many techniques to mitigate this problem whereas human “priors” are regarded as pure, rather than the result of previous experience that vastly reduce the true degrees of freedom or introduce look-ahead bias.

While sceptics won’t change their minds, many others are quietly ploughing on, utilising its benefits, some for their own private advancement and some, in the hope of the betterment of our society.

Two things

you realise when your computer fell off the bed and is damaged:

1. it’s hard to blog on a phone
2. you have an excuse to buy a new computer 🤔😄