Introducing Buzz: a turing complete concept for scaling Bitcoin to infinity and beyond

In this article I will outline an idea I have had for an infinitely scalable, Turing complete, layer 2 scaling concept for Bitcoin. I’ll call it Buzz.

Buzz is influenced by a number of ideas, in particular Ethereum’s VM, sharding, tree chains, weak blocks and merge mining. I’ll be discussing Bitcoin, but the Buzz scaling concept could be implemented on any PoW blockchain, and without Turing completeness.

How does it work?

Buzz is merge mined with Bitcoin and has its own blockchain called Angel which serves as a gateway (through two way peg) between the two different systems. In addition to merge mined blocks, Angel has a second (lower) difficulty which enables more frequent block creation, say every 30 seconds (weak blocks).

Buzz lifts its Turing completeness from Ethereum, taking much of its development but with a few adaptations to enable infinite scaling.

Ethereum’s current plans for scaling is through proof of stake with 80 separate shards. All shards communicate through a single master shard. Each shard has up to 120 validators who are randomly allocated to a shard and vote with their stake to reach consensus on block creation.

Buzz is quite radically different in its approach. It depends upon PoW which is elegant, resilient and has no cap on the number of consensus participants or minimum stake requirements. It avoids a split into an arbitrary number of shards with an arbitrary number of validators, and where all shards are homogenised and lacking diversity, such as different block creation times for different use cases.

In Buzz, each shard is called a wing and has its own blockchain. There is no cap on the number of wings, anybody can create one. Instead of making design decisions, the free market decides whether a wing will succeed or not. If there’s a wing with high transaction volume and high transaction fees, it will attract a higher number of miners and a higher level of security.

Buzz is basically a blockchain tree, since each wing can have multiple child wings, and the Angel sits at the top of the tree overseeing everything and communicating with the ‘other world’ that is the main Bitcoin blockchain.

The higher up the tree you go, the higher the hashrate will be, since a hashrate for a wing is equal to the hashing power of mining on all descendant wings combined with the mining on that wing but no deeper. Data and coins can be transferred up and down the tree between the parent and child wings. How frequently this can occur depends on the difference in hashrate between the child and parent. Nodes can operate and mine on any wing, but are required to also maintain the blockchains of any parent wings, up to and including the Angel.

The process of creating a wing is as trivial as getting a create wing transaction included in an Angel block. As well as being a gateway, Angel also serves as a registry, a little like DNS, keeping track of the properties of all wings.

If you want to create a wing with a block size of 1000MB and creation time of 1 second, that’s no problem. The market will probably decide your blockchain isn’t viable and you’ll be the only node on it.

Each wing will have its own difficulty level. If it was created with a 5 minute block target, its difficulty will be determined through the same process existing blockchains use.

There are no new coins created in this system, the incentive to mine comes from transaction fees.

The Angel has the hashing power of the entire Buzz network. Transactions here are the most secure, but also the most expensive. Angel itself has a limited functionality and conservative block size since this is the only blockchain that every full node is required to process.

How are wings addressed?

Every wing has its own unique wingspace address, a little bit like IP addresses and subnets.

For example, a wing that is a direct child (tier 1) of the Angel might have been registered with the wingspace address a9e. It might have a tier 2 child wing at the wingspace address a93:33, which might have a tier 3 child wing at a93:33:1a, and so on. If you’re familiar with subnets, this is like 255.0.0, 255.255.0 and 255.255.255.

There may be thousands of active and widely used wings, or there may only be a handful of wings which have a huge transaction volume. The free market will decide the tradeoff between hardware requirements, hash rate and transaction fees.

There may be geographically focused wings to benefit from lower latency, and micropayment wings which become the defacto standard for day to day use in a particular country. Travelling may involve moving some coins to participate on the wing where local transactions take place. Such routing could be automated and seamless to the end user.

Instead of trying to second guess a one size fits all solution for blockchain scaling, Buzz takes inspiration from the approach taken by the internet, where an IP address could represent anything from server farm to a raspberry pi, depending on the use requirements. The idea is to just create a solid protocol which enables the consistent transfer of coins and data across the system.

Different types of activity can take place in wings more suited to their use case. The needs of the network now may be completely different in the future, and allowing all wings to have a number of definable parameters and to be optimised for particular use cases (such as storage, or gambling) allows the network to evolve over time.

How are blocks mined?

Miners must synchronise the blockchain of the wing they are mining, and all parent wings up to and including Angel. Nodes can fully synchronise and validate as many blockchains as they like, or operate as light clients for ones they use less frequently.

In order to mine, hashes and blocks are created with a different method to Bitcoin.

The data that is included to generate a hash must also include the wingspace address and a public key.

Instead of hashing a Merkle root to bind transaction data to a block, a public key is hashed. Once a hash of sufficient difficulty to create a block has been found, the transactions data will be added, and then the block contents signed by the private key that corresponds with the public key used to generate the hash.

By signing rather than hashing blockchain specific data, we enable a single hash to be used in multiple wings (as it is not tied to a particular wings’ transactions) as long as it meets the difficulty requirement for each parent.

Since transaction data is not committed to the hash as in Bitcoin (where it is as a result of hashing the Merkle root), there needs to be a disincentive to publishing multiple versions of a block using the same hash signed multiple times. This can be achieved by allowing miners to create punishment transactions which include signed block headers for an already redeemed hash. Doing so means that miner gets to claim the associated fees, and the miner who published multiple versions of a block is punished by losing the reward.

When generating a hash, miners must include the previous block hash and block number for all tiers of wings they are mining on. This will allow all parents wings to have a picture of the number of child wings and their hash power.

Hashing in the wingspace a9e:33:1a, means that if a hash of sufficient difficulty was found, the miner could use it to create a block in the wings a9e:33:1a, a9e:33 and a9e. If the difficulty was high enough to create a block in the Angel, it means that wingspace will effectively ‘check in’ with Angel, and provide useful data so its current current hash rate can be determined and provide an overview of the health of wings. If a wing has not mined a block of Angel level difficulty in x amount of time, the network might consider the wing to have ‘died’.

If you had 30 second blocks in the Angel, over 1 million a year, that means even a wing with just 1 millionth (0.0001%) of the network hashing power should be able to ‘check in’ annually.

It is likely that this check in data will enable miners to identify which wings are the most profitable to mine on, and the network will dynamically distribute hash power accordingly. There will be less need to mine as a pool, since there will be many wingspaces to mine in, which should enable even the smallest hashrate to create blocks and earn fees. Miners can mine in multiple wingspaces at the same time with a simple round robin of their hash power.

Creating and modifying wings

When a create wing transaction is included in an Angel block, the user can specify a number of characteristics for the wing, such as:

  • Wingspace address
  • Wing title and description
  • Initial difficulty
  • Block creation time
  • Difficulty readjustment rules
  • Block size limit
  • Permitted Opcodes
  • Permitted miners

Hashing a public key and signing blocks with the corresponding private key allows us to do something else a little bit different: permissioned wings.

Most wings will be permissionless as blocks can be mined by anybody. However let’s say a casino or MMOG wants to create its own wing. It might want to do this so that it can have properties of a faster block creation time and it can avoid transaction fees by processing transactions for free, since they are the only permitted miner.

By only allowing blocks signed by approved keys, permissioned wings cannot be 51% attacked, and could even mine at a difficulty so low it is hashed by a CPU while retaining complete security. Users will recognise that in transferring coins into a permissioned wing, there is a risk that withdrawal transactions will be ignored, though their coins would not be spendable by anyone else so there is little incentive for doing so. It is up to them to decide whether the benefits outweigh the risks.

The property of permissioned block creation could be used for new wings which are vulnerable to 51% attacks due to a low hashrate. Permitted miners could be added until the wing was thought to have matured to a point where it is more resilient, and the permission requirement could be removed.

Angel transactions can also be created to modify wing parameters. Say changing to a different block creation interval at x block in the future. The key used to create a wing can be used to sign these modification transactions. Once a wing has matured, the creator can sign a transaction that waves their right to alter the wing any further, and its attributes become permanent.

How is this incentivised?

Transactions have fees. By creating blocks miners claim those fees. If you mine a hash of difficulty to generate a hash all the way up to the Angel blockchain, you collect fees on each level for all blocks you created.

There exists the possibility to add a vig, say 20% of transaction fees, to be pooled and passed up to the parent wing. These fees would gradually work their way up to the Angel blockchain, and could be claimed as a reward for merge mined blocks with the main Bitcoin blockchain.

Other ideas

Data and coins can be passed up and down the wings using the same mechanism Ethereum is planning with its one parent shard and 80 children topology. However there is no mechanism to pass data sideways between sibling wings, as sibling wings are not aware of each other.

I wonder however, if wings could be created to accept hashes from multiple block spaces. For example wing BB might accept also hashes, at say a 10 minute difficulty, from the wingspaces for AA and CC.

It would be possible to calculate the required difficulty level because all parent wings up to the Angel must be synchronised, and information about sibling wings will be included in block headers on the parent wing. This potentially creates a mechanism where wings can pass data between each other more efficiently and at lower cost, though I think there may be technical limitations with this system, in particular for transferring coins. This is because as far as the parent is concerned, coins transferred directly from BB to CC seem to have appeared out of thin air when passed back from CC to the parent, as the parent wing cannot see the activities of its children.

Thoughts on Proof of Work

Wings cannot merge mine with the main Bitcoin network without a hard fork so that it accepts hashes in the Buzz format.

This hard fork would enable the PoW to be shared, and offer increased security for both systems. Maintaining a separate proof of work between the systems, however, presents the opportunity to diversify the options available, such as Scrypt, SHA-256, X11 and Ethash.

If you wanted 30 second blocks on Angel with 4 proofs of work methods you could give each proof of work method its own difficulty targeting a 120 second block time.

When creating a wing, particular proofs of work could then be specified for use on that wing. Such as 20% Ethash and 80% Scrypt. This would open up PoW methods to the free market too.

Summary

  • Buzz is merge mined and distinct from the main Bitcoin blockchain
  • Recognises that there are infinite potential use cases for blockchains, with varying design requirements.
  • It is impossible to expect every node to process every transaction. There needs to be segmentation.
  • Attempting a one size fits all approach to scaling will lead to suboptimal and restrictive designs for many use cases.
  • Buzz creates a system where different segments with different properties can exist side by side and transfer coins and data, facilitating a free market where the best blockchains can thrive.

Thanks for reading through my idea, I hope the swirl of ideas buzzing around my head make sense now they have been converted into words. For any questions, thoughts or criticisms, please head to the comments.

What on earth is a Merkle tree? Part 2: I get more technical, but hopefully all becomes clearer

I previously written about understanding what Merkle trees are. If you haven’t read it, go and do so now.

I tried to keep it non-technical, and a keen observer would point out that the article better explained the benefits of hashing rather than of Merkle trees. I was trying to explain why Bitcoin benefited from Merkle trees, rather than how they actually worked.

In my previous article, the gist was that hashing allows you to verify that large quantities of data haven’t been changed using a hash, a much smaller amount of data. Merkle trees basically allow you to verify that a particular piece of data was present and hasn’t been manipulated, using only a small number of proofs rather than having to download all the data to check for yourself.

This time, I’m going to have another go at explaining Merkle trees, with the assistance of something we can all relate to… colours. I’m going to create what I’ve called a Merkcolour tree (see what I did there).

A Bitcoin transaction hash is the unique identifier for each transaction. In a block there are is as much as 1MB worth of transaction data. You can hash all the transactions in a block into a single 256-bit hash. However in order to prove that a transaction existed in a block you would have to download all the data used to create the hash, generate your own hash to check it is accurate, and then check transaction you wanted to verify was present in the data.

This would make it possible to store a copy of the 256-bit block hashes without having to store all the transaction data (upto 1MB) for each block – the down side is that in order to check a transaction is present you have to download all the transactions data instead of a small number of proofs, which is much more quicker and efficient as Merkle trees enable.

So how do Merkle trees work?

Well, the unique Bitcoin transaction id, which is actually a hash created by hashing the transaction information together, looks like this:
cf92a7990dbae2a503184d6926be77fc85e9a9275f4222064ee78eeb251d36b2

And if you combine (concatenate) two transaction ids back to back:
cf92a7990dbae2a503184d6926be77fc85e9a9275f4222064ee78eeb2 51d36b2d8f4744017dc79f8df24e2dba7fd28e5fd148c3b01b5f76dede8ef3ac4e5c340

That combined data can be hashed together (using the SHA-256 method) into the following hash:
474a1d00a80dd927ba87404371c11c7db24bc58b0a712ffacdb09a47dc1bec89

Instead 512-bits of data for both transactions, the hash is 256-bits of data.

Now lets adapt this logic to colours. Each colour is represented in the same hexadecimal format as a hash, it’s just that a 256-bit hash is more than 10 times longer than a 24-bit colour.

As an example, red is #ff0000, blue is #0000ff.

If we combine those colours together we get #800080.

Instead of 48-bits of data for both colours, the combined colour is 24-bits of data.

A Merkle tree is basically a process by which pairs of hashes are merged together, until you end up with just one, the root. This is best demonstrated with colours in the image below (click it to open).

Merkcolour tree

In the image, we start with 16 different colours (labelled A to P) – these are the leaves. Each colour has been paired with a neighbour and combined together to create a branch. This process is repeated as many times as necessary until you end up with one final colour – the root.

Now, the colour (leaf) we’ve labelled I in the diagram is #ff0000.

If, instead of creating a tree, I simply hashed all the colours together the outcome would be as follows:
#000064#007777#007700#777777 #1f1fff#5cffff#47ff48#ffffff #ff0000#770000#f07000#614600 #ff21b5#ff1f1f#ffff00#d7c880

Hashes (using the MD5 method) to:
8c4878686b62656fbe81d50c3a832728

If I provided you with that hash, and told you it included the colour #ff0000, the only way I can prove it is by sending you all 16 colours in that order (384-bits of data, removing the #) so you can generate and confirm the accuracy of the hash for yourself.

However, because we’ve created a tree, if you know the root is #8e7560 (a product of all leaves – ABCDEFGHIJKLMNOP), we can confirm that #ff0000 (I) was included using only 4 proofs:

  1. #489194 (ABCDEFGH)
  2. #f58255 (MNOP)
  3. #a95b00 (KL)
  4. #770000 (J)

Let’s start at the top and work our way down to the root:

#ff0000 (I) (we want to confirm) when combined with 4) #770000 (J) gives:
#bb0000 (IJ) which combined with 3) #a95b00 (KL) gives:
#b22e00 (IJKL) which combined with 2) #f58255 (MNOP) gives:
#d4582b (IJKLMNOP) which combined with 1) #489194 (ABCDEFGH) gives:
#8e7560 – which is the correct root!

If we ended up with any other value than the root, we know some of the data we have been given is inaccurate and cannot be trusted.

This means that instead of hashing all the colours together and needing to download 384-bits of data to confirm its accuracy, we are able to download just 4 proofs, or 96-bits of data.

The efficiency gets even bigger the more leaves you have, as each time you double the data (number of leaves), you only add one additional branch which is one extra 24-bit proof for colours, or 256-bit proof for hashes.

For example here’s how much proof data is required to verify the following:
32 colours (768-bits): 5-proofs (120-bits, 84% efficiency)
64 colours (1536-bits): 6-proofs (144-bits, 91% efficiency)
128 colours (3072-bits): 7-proofs (168-bits, 95% efficiency)
256 colours (6144-bits): 8-proofs (192-bits, 97% efficiency)
512 colours (12288-bits): 9-proofs (216-bits, 98% efficiency)
1024 colours (24576-bits): 10-proofs (240-bits, 99% efficiency)

A key distinction of hashes and colours is that hashes are one-way and unpredictable. That means you cannot work out what two hashes were combined to create a hash. The opposite is true for colours, if you know a colour it is possible to work out exactly what combinations of colours could have created it. The Merkcolour tree is only useful for visually demonstrating the concept, if you could reverse engineer a hash in the same way you can a colour it would not be reliable.

Hopefully this makes sense! The key take home message is that instead of having to download an entire 1MB block to confirm that a transaction was in it, you’re able to download a small number of proofs for validation. This makes it a lot easier to verify transactions without having to keep a copy of the blockchain or download large quantities of data, for example on devices such as smartphones where this is less practical.

Merkle trees make the process of verifying data hugely efficient, and while Bitcoin could exist without them, it would require an awful lot more resources such as processing, bandwidth and storage – and running clients on mobile devices would be far less viable.

Thank you Ralph Merkle.

Unintended consequences: Could proof of stake just become no proof of work?

Bitcoin operates through a process known as proof of work (PoW). In order to determine which network participant gets to create the next block (and claim a reward), the process requires the contribution of computer processing power. The more processing (work) you perform, the more likely you are to be rewarded with Bitcoins.

Running this hardware is very expensive, the Bitcoin network is already said to consume as much electricity as the entire country of Ireland.

Satoshi Nakamoto’s vision when he created Bitcoin was that everybody would mine Bitcoin on their computers, all around the world, and that this would decentralise the network.

Unfortunately, CPUs are incredibly inefficient miners. A decent laptop might manage around 14MH/s. A specially designed (ASIC based) AntMiner S9 can achieve 14TH/s – that’s 1,000,000x faster.

Nakamoto could not have foreseen the rise of ASICs when he wrote the Bitcoin white paper. Consequently, instead of being distributed around the world, Bitcoin has faced huge centralising pressure. The number of people required to control Bitcoin can fit around one table. Centralisation provides self perpetuating benefits of easier access to the best hardware and cheapest electricity, though once ASIC chips bump up against Moore’s Law there’s good reason to believe we will see a shift back towards decentralisation.

The holy grail of cryptocurrency would be the security of proof of work, but without the cost and centralisation. I first read about proof of stake (PoS) a number of years ago and, seduced by the idea, immediately invested in PeerCoin, the first cryptocurrency to implement it.

So what is proof of stake?

PoW uses expensive and ‘wasteful’ electricity to try and calculate a hash of sufficient difficulty for the network to accept – enabling that participant to create a new block.

PoS works the other way around. There are a number of proposals, but the basic principle is that each participant can ‘stake’ their coins to create a kernel (type of hash). The bigger the stake, the bigger the chance their kernel will ‘match’. Match what? Well, the blockchain itself generates a random and unpredictable seed based on the data in the proceeding blocks (also by hashing), and the closest matching kernel gets permission to create the next block, and is rewarded for doing so.

As there is no requirement to lock up computer processing power, everybody can run the software on their own machine without the expense and hardware requirements of PoW.

Sounds great, doesn’t it? Well, as with the unintended consequences of PoW, let’s try and foresee how the PoS landscape might evolve.

Under PoW we have seen the rise of pooled mining. Pooled mining has been wildly popular because it makes mining income more predictable.

Think of PoW like a lottery. The more processing power you contribute, the more tickets you get. In Bitcoin there is just one winner every 10 minutes.

If the current bitcoin difficulty didn’t increase, even with the most efficient miner – the 14TH/s Antminer S9, you’d have to enter this lottery for over 2 years on average to win just once.

If you join a pool that has 25% of the hashing power (lottery tickets), then you can expect that pool to win once every 40 minutes on average, and you can then regularly collect your share of the winnings. This is favourable as opposed to running your hardware for years in the uncertain (unlikely) hope of winning the jackpot. Pooled mining in PoW offers no other benefit than making your income more predictable.

Would the same be true of PoS?

There has been testing in PoS experiments that has gotten the block creation time down to 3 seconds per block. This means instead of having 52,560 lottery winners per year in Bitcoin, you could have 7.6 million winners each year. This would certainly reduce, though not eliminate, the appeal of mining pools.

However, in cryptoeconomics we must assume that each participant will always act in their own self interest. Could there be other benefits from PoS pooled mining that are not present in PoW?

In digital security, randomness is very valuable. In PoW the randomness that selects the next block is generated by an external source – all that hardware calculating trillions of random hashes. In PoS this necessary randomness does not come from an external source, it can only come directly from the blockchain itself.

This means a seed generated from previous blocks is used to determine which participant will create the next block.

There are two different data sources you can hash for this randomness. If you included all the contents of the block to generate a hash, this would be a disaster, since there are infinite combinations of block contents. If it was an individual’s turn to create the next block and they had sufficient hardware they would just crunch as many combinations of block contents as possible and hopefully find one that generates a seed matching a kernel they control, allowing them to create the next block and repeat the process again.

This ‘stake grinding’ wouldn’t represent a shift away from proof of work, it would just mean work has taken place but without any proof or transparency.

An alternative option is to only hash header information which cannot be manipulated, such as the block creator’s signature. A potential issue here this is that if you pooled together, you could gain a competitive advantage.

Imagine you’re in a pool with 30% of the staked coins. This should mean that your pool creates 30% of the new blocks. However, let’s speculate an instance where the seed to determine the next block has two pool members as the two closest matches. Imagine the closest match signing a block would create a hash that would allow the next block to be created by a non-pool member, whereas the 2nd closest match would allow the next block to be created by another pool member. If you had sufficient hardware the pool could work to rapidly calculate the best combination of block creators to maximise revenue for the pool.

You can try to mitigate this risk by punishing participants for not creating a block when it’s their turn, but getting the economic balance right to not overly punish people with less reliable Internet connections for example (another centralising pressure) strikes me as an unenviable task.

Ultimately, if the pool has the size and hardware resources to crunch the numbers far enough ahead – it’s still going to game the system when it calculates a combination that will likely generate 10 consecutive blocks, compensating those members who lost out in the process for the greater benefit of the pool.

Such a system could actively incentivise centralisation. The bigger the pool, the greater the advantage. It could create a race to the bottom, since while everyone may recognise this centralisation as undesirable, they also must make an economic sacrifice to avoid participating in it.

Perhaps this centralisation pressure and obscuring of work would be an unintended consequence of PoS. All I know is, the more I study PoS and its goal to provide the security of PoW without the cost, the more a phrase from growing up in Yorkshire comes to mind… “you don’t get owt for nowt”. In other words: there’s no such thing as a free lunch.

What on earth is a Merkle tree? A non-technical answer the question you’ve always wondered

Merkle trees are one of the areas of Bitcoin I found hardest to understand.

The biggest problem was every time I read about them I just saw diagrams that didn’t make any sense with lots of multiplication and layers. Ignore the terminology, don’t worry about leaves, branches, roots or trees – these are words concerned with the technical process of creating one and will just confuse you.

Perhaps more helpful than understanding exactly how a Merkle tree is created, is understanding why they are useful. For the techies, I apologise for the over simplifications.

If you haven’t already, check out my article where I explain hashing. Hashing is such a simple concept, and it has very cleverly been integrated into Bitcoin in a number of ways.

Basically, a hash is a way to convert any information at all into a completely random but consistent value.

There are many different hashing methods, Bitcoin uses SHA-256 which enables you to convert any data into a 256 bit (32 byte) value.

If I hash the text ‘seebitcoin.com’ I get the 256 bit hash: e37695b5a5f8671d24e4160ec69755f73061bbc01d319403ce5ba5034bd57dc0.

If you do the same you will get the same result. A tiny change and the entire hash will be different, for example ‘seebitcoin.nom’ always gives the hash: cebe400cd1a7319a6c1e881d85322120f03c95c4dc00d996b1ae3472398753fa.

This allows us to do some really useful things. You can take the entire contents of a Shakespeare book and convert it into a 256 bit hash. If you have a hash for all your Shakespeare books, and your friend gives you a hash for their favourite Shakespeare book, you can know whether you have exactly the same copy.

If anything had changed, even as small as say an uppercase letter becoming lower case, the hash would completely change and you could know for certain the books were not identical.

We could take the complete works of Shakespeare, and for each piece of work we could create a hash. This means to have enough information to be able to verify 884,647 words, we’d only need 196 hashes total (38 plays, 154 sonnets and 4 poems).

This verification information could be stored on a single page of paper, while the complete works of Shakespeare could fill a bookcase.

While people probably aren’t too worried about the words in their copy of Hamlet being tampered with, in the Bitcoin world people want to know with certainty that none of the previous transactions they have stored have been altered, as this would introduce the possibility of fraud.

Like the complete works of Shakespeare, the Bitcoin blockchain is huge. There are over 420,000 blocks which can contain as many as 1MB worth of transactions. Currently the blockchain is over 75GB in size.

Just like all the words in a Shakespeare book can be converted into a 256 bit hash, all the transactions that took place in a block can be converted into a single 256 bit hash. This hash, in Bitcoin terminology, is called the Merkle root (the final value when calculating a Merkle tree).

This means for 420,000 blocks, you could have enough information to verify with certainty every transaction with just 13MB of data. Considering that’s over 75GB worth of transactions this is extremely useful.

In a Bitcoin block, the header data (all the important information about the block) and the transaction data (all the actual transactions, the bulk of the data) are separate.

This means that the software only needs to use the data in the block headers to do most things, which is far more efficient. Another benefit is that your smart phone only needs to download the block header data (around 32MB for 420,000 blocks) to run a Bitcoin client, and can just request additional data as needed without having to download and store an entire copy of the 75GB blockchain.

Merkle trees are what makes these things possible, and are basically just a method to convert all of a block’s transactions into a single hash to be used to verify no information has been manipulated.

Any requests for other explanation articles? Let me know in the comments.

Correction: I erroneously suggested a hash was 256 bytes instead of bits, values have now been corrected.

Let’s talk hard forks: the most exciting area of cryptocurrency?

In my last article I outlined the difference between and soft and hard fork. Now, I will switch my focus to the different types of hard fork.

Not all hard forks are born equal

A hard fork is when a blockchain splits into two separate and incompatible versions. The reasons for this happening and the consequential fallout are varied.

At the uncontentious end of the spectrum, you have essential hard forks. In its early days, a simple overflow bug in the then not properly audited Bitcoin code allowed somebody to create 184 billion Bitcoin out of nothing in a single block. The protocol allows a maximum of 21 million Bitcoin to ever be produced, and so this bug violated the protocol and rendered that version of the software useless. Bitcoin basically crashed. An update was essential to fix the problem, and a new version was released within hours.

Fixing critical bugs is as uncontroversial as it gets – since crypto economics assumes every participant should rationally act in their own self interest there were no participants who would benefit from Bitcoin remaining broken and the forked blockchain quickly became the dominant one.

Also uncontentious is an upgrade hard fork. Bitcoin hasn’t had one of these yet, but Ethereum has. These involve improvements to the protocol. In the case of Ethereum it was established from the beginning that the protocol would hard fork 3 times, gradually introducing new features. Those using Ethereum accept as part of its use they will need to upgrade their software in order to stay on the main blockchain, and since they are integral and promised improvements they are eagerly anticipated.

The next type of hard fork gets a little more ambiguous. There is consensus in the Bitcoin community that the 1MB block size needs increasing through the use of a hard fork. Where the level of contentiousness exists is when and by how much this increase to the transaction capacity needs to be. There have already been a few failed attempts to hard fork to a bigger block size. Most recently Bitcoin Classic failed to gain enough support to introduce a 2MB block size hard fork, requiring but not obtaining support of 75% of miners be running the new software in order for it to be activated.

Since even Bitcoin Core has an increase to 2MB blocks on its scaling roadmap for the future, it is very unlikely that if such a fork had been activated, the other 25% would have remained on the old blockchain stubbornly insisting that they didn’t want to upgrade ‘yet’, as they would simply be left behind as the wider community accepted consensus had been reached.

If a fork to permit a 100GB block size increase had been successfully activated things would be different. There are many people who are strongly opposed to the increased centralisation, reduced protection from DoS/spam attacks and lack of fee incentive that larger blocks could bring. It is likely that no matter what a portion of the community would reject the fork and continue to participate on the original blockchain.

Here, we have reached the crux of what a contentious hard fork is: ideological.

Sometimes, consensus is simply impossible to achieve. If you passionately and ideologically believe something is right, you’d rather continue using and supporting the vision you believe in, even if you’re in a minority of <1%.

The ‘worst’ type of contentious fork would involve a community truly split down the middle, 50/50, as neither chain could be said to have won, and you’d find yourself with a format war, two competing and widely used solutions waiting for a winner to emerge. Many view this as undesirable.

There is no reason two sides of a hard fork cannot coexist peacefully, and both can be traded on exchanges with people free to use whichever fork they believe in. There are some complications though.

In the event of a hard fork, anybody who owns coins at the time the hard fork occurs will own those coins on both blockchains.

A hard fork will probably seen as a negative initially and so cause the value of the coins to be lower, though this will probably be priced in once it is apparent the hard fork will occur. Then, after the split, the value of each set of coins would reflect the mining and community support for each side of the fork.

If you log into an exchange which has decided to support both sides of the fork, you will find you have two balances. If you’re with an exchange that has decided to support only one side of the fork, you’re possibly going to miss out and lose some coins that could hold, even if small, a future value. I wouldn’t be surprised if those exchanges left themselves open to legal action over the missing coins.

It is a simplistic and highly speculative example, but let’s say a hypothetical $100 coin splits into two blockchains with a 75%/25% hashing power split, you may see the overall value drop 80% leaving the coins valued at $60 and $20 on each side of the fork respectively.

While it will be immediately obvious how the miners have split their support between the chains, it is a lot more difficult to estimate community support. If you had a majority of miners in favour of fork A, and a majority of the community in favour of fork B, you’d likely see a quick swing in price with fork B overtaking A in value. There will likely be arbitrage opportunities for anybody who is able to identify the likely disparity between miner support and community support which is impossible to see until left to the market to decide at the exchanges.

Cryptoeconomics anticipates that people will act in their own self interest. If someone genuinely believes in one side of the fork ideologically, it is likely they will ‘dump’ their coins on the losing side of the chain in order to lower its value and in theory increase the value and the likelihood of success on the chain they believe in.

This is a gamble, they could dump all the coins they don’t believe in, and then find out the community valued that side of the fork more highly leaving them ultimately with a lot of highly devalued coins. It is also possible such a devaluation would be temporary, and if people persist to support the ‘losing’ side its value could recover.

In the case of Ethereum, which is about to undergo the first ever contentious hard fork, there are people who strongly believe in both sides of the fork and would be prepared to gamble dumping the coins they oppose to negatively influence the price.

Hard forks also create an unintended problem, replay attacks. If you have two almost identical protocols, the format of the transactions you submit to the network are identical. If you see a transaction on one side of the fork sending money from X to Y, anybody can view it and submit that same transaction to the other blockchain so that it occurs there too, even if the people initiating the transaction don’t want that to happen.

There are ways to try and prevent this, but it adds a layer of complexity and is a barrier to peaceful coexistence. If you have a lower value coin you want to send to somebody, you could ‘lose’ your higher value coins in the event of a replay attack, so this factor likely favours the most popular (highest value) side of the fork, as people want to minimise the risk of losing their more valuable coins.

Ultimately, the safest reaction and likely most common response to a contentious hard fork is to wait and see how it all plays out.

If you had $100 coin, that split to $60 and $20, it may be that if you wait a week the coins are worth $79 and $1 – overall you’ve not lost anything. The sooner you move your coins, the bigger the risk you face, but also the bigger the potential reward.

Many people will expect that on the smaller side of a fork there will be a huge dump of coins from people who only want to hold a balance on the ‘winning’ side of the chain. This could have a knock on effect on the viability of mining, with only those of the strongest ideological resolve mining at a loss in the hope of future returns.

Let’s look at Ethereum’s imminent hard fork as an example, which will be fascinating to observe. The community is split following an attack where an individual was able to steal a large number of tokens from a smart contract called the DAO that huge swathes of the community had invested in. This was no fault of the protocol or Ethereum itself, but rather a badly coded contract. Ethereum has marketed itself as immutable and “contract is law”, so there is an ideological argument that a hard fork to return the stolen funds from a badly coded contract undermines the entire project, which is why a number of the community are so strongly opposed.

Once the hard fork takes place, there are many people who oppose the hard fork and want to remain on the original chain who do so acknowledging there is a possibility the vast majority of the community will dump their coins causing its value to plummet. This seems in violation of the principle that people would always act in their self interest as they could see the value of their own coins diminish and mining become unprofitable.

Rather than a bad thing, many of these people see the dump as a great time to buy these coins at a heavily discounted price, with the opinion that if something like the DAO hack has happened before, it will likely can happen again. They envisage a future where the community accepts sacrificing its immutability was a mistake and hard forking to solve problems is not viable, and that people will abandon the compromised chain and come back to realise the value of the original immutable, “contract is law” blockchain and the value of their holdings will increase finally allowing them to profit. It’s a long term hedge.

A contentious hard fork outcome is so hard to predict, they’re quite a lot like an election. There are certain indicators, but until it happens there’s always the possibility of a surprise. In reality nobody knows what will happen, but they are not the end of the world, and they are rather exciting.

Understanding Bitcoin: what’s the difference between hard and soft forks?

It occurred to me at a Bitcoin meetup the other week that forks are one of the concepts of cryptocurrency that can be a source of confusion, so I thought I’d have a go at explaining the basics. It is a particularly interesting time for forks as the Ethereum network looks set to go through the uncharted process of a contentious hard fork, but more on that another time.

What is a fork?

Bitcoin and other cryptocurrencies are distributed networks.

What’s incredible about them is that they operate on thousands of different machines with nobody in charge, but are still able to reach a consensus.

Bitcoin is basically a giant list of transactions – every transaction that has ever taken place on the network in fact. Every 10 minutes, all the transactions from the previous 10 minutes are collected together into a block, and then this block is added to the end of the chain of all the other blocks which contain all the previous transactions – the blockchain.

There are two roles involved in distributing the network, miners and nodes. Nodes basically just connect to the network and share blocks and transactions with other nodes. Miners have the additional responsibility of creating blocks, and are rewarded with new Bitcoins for doing so. (For an explanation on how the network decides which miner will create the next block, see my article on proof of work.)

In order for all the machines to work together, they have to operate according to a strict series of rules. This particular group of rules together are known as the protocol.

An example rule of the Bitcoin protocol is that a block can contain a maximum of 1MB worth of transactions.

Remember, every participant on the network has a copy of exactly the same rules/protocol. If a miner tried to create a block that contained more than 1MB of transactions and then sent that to other nodes, they would simply say nope, that’s not valid – and then they would discard it instead of passing it on so it can propagate around the network. It would be completely pointless to create such a block, a waste of processing power.

Sometimes people believe improvements can be made by changing some of the rules. Some people in the Bitcoin community would like the block size increased from 1MB to 2MB so that the number of transactions can double. This would require a change to the rules of the Bitcoin protocol and could only be achieved through what’s called a hard fork – everyone would have to upgrade their software to the new protocol rules.

There are other changes that can be made that involve the enforcement of new rules, but the changes do not require a change to the protocol that everybody agrees upon. For example, if all the miners said they were going to mine blocks with a maximum size of 0.5MB – everybody on the network would accept these blocks as valid since they fall within the protocol’s 1MB allowance.

If over 51% of miners all agree to a maximum block size of 0.5MB they can force this change upon the entire network without anybody else having to change their software. This is called a soft fork. Every node and miner will accept the blocks as valid and build on top of them.

You might think if 49% of miners were still creating 1MB blocks, surely the blockchain would have some 0.5MB blocks and some 1MB blocks, since they are all technically valid within the protocol and recognised by all participants as legitimate.

It could work like that, but in the case of a soft fork it doesn’t. 51% is the magic number at which point the majority of miners can force all other miners to limit themselves to 0.5MB blocks. Since miners get to choose which blocks they build upon, the 51% of miners could simply ignore any blocks they no longer considered valid within the new rules they have implemented, and so only 0.5MB blocks would ever be included in their blockchain.

Some people argue a soft fork is a confusing term because the network itself doesn’t really fork (split in two), and all software would still follow the same blockchain. It is however a fork in the sense that miners who hadn’t upgraded their software would find themselves building incompatible (forked) blocks, it’s just that those blocks would be ignored and consequently orphaned by other miners and would quickly become irrelevant.

Technically, a soft fork is exactly the same as a 51% attack, and some argue it should be described as such. I think the big distinction is that soft forks generally have a social consensus and are accepted as improvements to the network, while a 51% attack is widely considered to be harmful. An example 51% attack would be to include no transactions (0MB) in any blocks, as is permitted, and cause the network to grind to a halt. In fact, many would argue the example I gave of of lowering the block size to 0.5MB and consequently halving transaction volume is better described as a 51% attack than a soft fork, but it was easier to explain than actual soft forks widely considered improvements such as P2SH and Segregated Witness.

In summary, a soft fork involves a change to the rules that only minors must agree upon and implement, a hard fork involves a change to the protocol that every participant must agree upon and implement.

There are currently over 5,600 Bitcoin nodes, while only 14 different mining pools have found blocks in the last month. This means soft forks are a lot easier to implement, 400x easier in terms of a rather simplistic count of the number of installations that need their software upgraded.

It’s not quite that simple though, consensus is a lot more fuzzy and complicated. In my next article I will talk more about hard forks, which open up a whole new jumble of exciting possibilities and unintended consequences. Stay tuned.

photo credit: Fork via photopin (license)

An objective look at past Bitcoin rallies and what we might expect now

I’ve been following Bitcoin since 2011. The current price rally feels very familiar. ‘Feel’ is not good enough for me, I want to be objective – look at historical trends, develop a hypothesis, and then allow the passage of time to determine its accuracy.

I am not a trader. I view it as gambling and I plan to keep my Bitcoin long term without regard for the price… a classic HODLer. I don’t get excited about fractals and prices, I get excited about BIPs.

Despite this I was curious to objectively look at previous rallies to see if there were any consistent trends, and then to speculate on what these trends would mean if they held true again, which I am not suggesting they will.

I decided to look at the peak of each rally, and then look for measurable points of reference on either side. I found I got useful information looking at the 1 and 2 month points before the peak, as well as a value 4 months after the high to see where the price had settled.

I’ll use the most conservative figures from each rally, and use the lowest recent price from the last month before the current rally began to make a projection – $437 on 19 May.

Here are the previous 3 big rallies:

8 Apr 2011 = $0.75
8 May 2011 = $3.87 (+420%)
8 Jun 2011 = $29.60 (1 mo: +660%) (2 mo: +3850%)
---
8 Oct 2011 = $4.01 (+435% from start, -86% from high)

9 Feb 2013 = $23.65
9 Mar 2013 = $46.85 (+100%)
9 Apr 2013 = $230 (1 mo: +390%) (2 mo: +870%)
---
9 Aug 2013 = $93.36 (+295% from start, -59% from high)

4 Oct 2013 = $121
4 Nov 2013 = $225 (+86%)
4 Dec 2013 = $1147 (1 mo: +410%) (2 mo: +848%)
---
4 Apr 2014 = $449 (+271% from start, -61% from high)

Using the most conservative figure at each stage gives the following projection:

19 May 16 = $437
19 Jun 16 = $812 (+86%)
19 Jul 16 = $3979 (1 mo: +390%) (2 mo: +811%)
---
19 Nov 16 = $1621 (+271% from start, -59% from high)

A lot has changed in the last few years. There is far more interest from China and a lot more liquidity in the markets, but humans are still humans. In the past we’ve gotten carried away by the rising price, created a bubble which then has then burst before the price settles again at a higher than starting level following a correction.

Those numbers look and feel really high now, but so did the previous rallies. I’ll be interested to see whether this pattern plays out again.

Understanding Bitcoin: the childhood game that rules the network

Its a childhood classic, “guess what number I’m thinking of”. Its funny to think, but Bitcoin is basically a computerised version of that game.

In the childhood game, you may play with a friend and state that the number is between 1 and 10: in this case each round of the game won’t last long. How could you make the game longer? Easy, by increasing the difficulty to make them guess a number between 1 and 500.

Let’s say ideally you want a game to last for 10 minutes, and you have 5 players. If each player can make 60 guesses per minute, in 10 minutes each person will make 600 guesses, that’s 3000 between the group. Double this number and you know that for the game to last around 10 minutes, your target difficulty is a number between 1 and 6000.

This simple game has advantages, as long as nobody can know what number you might be thinking of, it is impossible to cheat. The only advantage that can be gained is through guessing more quickly than the other players. The more guesses you can make, the more likely you are to win.

In Bitcoin this game is played is to decide who processes the last 10 minutes worth of transactions. The winner creates a block of transactions, adds it to the blockchain, and is rewarded with (currently) 25 Bitcoins.

By playing a game that is impossible to cheat, it prevents attacks against the network, as anybody who wants to maliciously attack the network would need to be able to make over 50% of the guesses.

In Bitcoin terminology, the number being guessed is called a nonce, and guessing is called hashing. A hash rate is basically a guess rate.

Hashing sounds complicated, but it really isn’t, its just a way to turn any information into an unpredictable sequence of letters and numbers.

As an example, I’ll invent a simple hash method that converts any number into a completely different but unpredictable number.

For my method, you prefix the chosen number with 98765 and then divide the number by 4321. You then multiply the 1st-5th by the 6th-10th digits after the decimal point to get a hash.

This is completely made up, so don’t worry if you don’t follow, but it would produce the following demonstration hashes:
45 = 6660386432
(Method: 9876545/4321 = .7081694052 = 70816 * 94052)
100 = 2955753
(Method: 98765100/4321 = .0006942837 = 00069 * 42837)

This would be a disastrous, collision-ridden hash for Bitcoin, which uses a far superior method called SHA-256 – but it demonstrates the principle that you can turn anything into an unpredictable hash that everyone can independently verify for themselves.

With my hash, as with Bitcoin, I can set a difficulty level. Let’s say I require the first 5 digits of the hash to be ‘11111’ (Bitcoin uses zeroes).

With my method the number 74340 would produce a hash of the target difficulty:
74340 = 111110100
(Method: 9876574340/4321 = .9595001158 = 95950 * 01158)

This cannot be cheated, you can’t predict a number which will produce a hash beginning ‘11111’. You (your machine) simply has to keep guessing until finding one that successfully matches. This is where ‘proof of work’ comes from – finding a matching number proves you have done work and once you’ve guessed correctly you can tell everybody your number and they can check it for themselves.

Back to our kids ‘guess the number’ game there is one common problem: cheating. If your friend guesses correctly but you don’t want to relinquish control you can say ‘wrong’. If you pick a number and tell your friend what to guess – they can secure victory on their first guess. This would have disastrous implications for Bitcoin where the reward is the currency itself.

Hashing solves the problem of who is in charge of deciding the number because there is no number, just a difficulty level – everybody just keeps guessing until they find a hash that satisfies it and they get to broadcast their proof and claim their reward.

The Bitcoin network constantly adjusts its own difficulty level so that on average one correct guess is made (hash is found) every 10 minutes.

The processing power that goes into generating these hashes is mind boggling. A group of researchers tried to estimate the number of grains of sand in the world – every grain of every desert and beach. Their estimate was 7.5 x 1018 grains of sand – seven quintillion, five hundred quadrillion.

Currently the Bitcoin network takes just 6 seconds to make as many guesses as their are grains of sand on earth and only one of those guesses is correct every 10 minutes. That’s one hell of a game.

Craig Wright is a liar and anyone who still believes he is Satoshi is a gullible fool

Forgive the blunt tone of this post, but I wanted to be to the point. Craig Wright has now declared he will not be providing the promised proof that he is Satoshi.

I thought he may try and drag it out a little longer… but the outcome was always going to be the same – failure to provide evidence.
His ruse didn’t play out how he planned. He thought he could gain credibility from a simple confidence trick on Gavin and others and that the community would all hail Caesar and accept a public signature from block 9 as proof. He was naive and/or desperate to think that is how it would play out.

Let us be objective. Craig Wright’s reputation and credibility is mud and could not sink lower. Consequently he has absolutely nothing to gain by failing to provide the proof he claims to possess. The only objective explanation is that he is a fraud.

His only shred of hope now is that a small core of gullible fools will continue to believe in and defend him, don’t. He has made this mess himself, and if he is telling the truth only he has the power to fix it – do not drag your reputation down with his. Even if he somehow did turn out to be Satoshi he is responsible for how this has played out.

The proof that he is a fraud will likely come in the outcome of his investigation by the Australian Tax Office.

On a human side, Craig Wright must be in a pretty desperate place to even try and pull this off, I actually feel bad for him. I hope now that he will face reality and come clean with the full truth, for his own sake and for the family of Dave Kleiman.

The feather in Wright’s cap: demonstrating a signature from block #1 to Gavin Andresen is going to backfire spectacularly and will ultimately prove his downfall

Imagine you’re trying to convince the world you created Bitcoin.

What greater coup than convincing Gavin Andresen of your legitimacy, Satoshi’s chosen successor and the man who arguably knew him better than any other.

So, how would you do it? Well, if you have a private key that the real Satoshi Nakamoto would possess it’s easy. You ask for a message to sign, sign that message, and then send the signature back to be verified. The whole process can be completed from the opposite ends of the world in minutes with 100% certainty.

What would you do if you didn’t have that private key? Well, you might ask someone to fly to London to meet with you and ask them to sign a nondisclosure agreement. That way, if the meeting doesn’t go as you hoped, they’re not allowed to comment on it.

You could then alter the software on a laptop to make it say that an invalid signature is actually valid. You could then stick a factory seal on the box, bring it out, and claim your assistant just bought it brand new from the shop so it could not possibly have been interfered with. You could then demonstrate an invalid key, and then take away the USB pen and laptop so that person had no evidence to check for themselves.

Somehow, that plan worked. The BBC ran with the headline “Craig Wright reveals himself as Satoshi Nakamoto”.

Then, everyone else was confused – where was the evidence?

Craig Wright has a big problem now. He could have tried to claim to be Satoshi Nakamoto but concoct some elaborate excuse for why he was unable to access any private keys that Satoshi would have owned. This would have made an already difficult con even more insurmountable, but would at least give him plausible deniability – nobody could prove his story was false.

The problem is, Craig Wright claims to have demonstrated using a private key from block #1 to Gavin Andresen. That means he can no longer concoct such an excuse as a get out of jail free card. He now must either put up or shut up.

If Craig Wright genuinely signed a message from block #1 for Andresen, then he still has that private key and can prove it in minutes. That he keeps dragging this out is very telling.

Convincing Gavin Andresen he possessed the private key to block #1 once seemed his greatest asset, it has now become his greatest liability.

photo credit: Feather in my cap via photopin (license)