- Title: LN chapter 7 - Normal operation of a pre-taproot LN channel - Summary: --- Table of contents ### Overview In this chapter we will cover the normal operations of a channel. This involves understanding how HTLCs are added to a channel and how channel peers commit to a new state to include these HTLCs. We will then cover how a channel's normal flow is re-established after a disconnection. Finally, we will cover the cooperative channel closure flow. By the way, all of these topics are covered in [BOLT2](https://github.com/lightning/bolts/blob/master/02-peer-protocol.md). ### Preliminaries Here are some concepts and flows that we have covered previously in our series that might be useful for understanding this chapter. Don't be afraid to go back and reread them if you need a reminder. - [Updating state]: why channel peers have asymmetric state and how they update this state in a trustless way. - [HTLC deep-dive]: what HTLC outputs look like on a commitment transaction. - [Opening and announcing a pre-taproot LN channel]: how two nodes open a channel.  ### Normal channel operation #### Set up Alice and Bob have successfully opened their channel. They have both seen that the funding transaction is confirmed and they have exchanged the `channel_ready` message with each other to indicate that they are ready to use the channel. The state of their asymmetric commitment transactions currently looks like this: ![1-initial](https://cdn.satellite.earth/96ea96228ee4ba11199405503b1ad5e0c3c07e913bafd7b0217f0b35197d14d1.png) Both commitment transactions spend the 2-of-2 multisig funding transaction output. Since it is a 2-of-2 multisig, spending from this output requires signatures from both peers. The signatures are represented by the two boxes at the top of the commitment transactions. You can see from the diagram that Alice's commitment transaction has a signature from Bob represented by the blue box; Bob has a signature from Alice for his commitment transaction represented by the green box. Each commitment transaction also has the `to_local` and `to_remote` outputs which pay the respective parties their current channel balance. Both Alice and Bob can sign their own commitment transactions at any time and broadcast it to the Bitcoin Network. This woul be a **force close**. For the sake of making the next few diagrams easier to follow, we will ignore the funding transactions as well as the `to_local` and `to_remote` outputs for a while since our initial focus will be on adding and removing HTLC outputs. So the above diagram is now simplified to this: ![2-simplified](https://cdn.satellite.earth/980d403ccb67840ce726489611fff5f8225312c06ba254d96dc4bf483152572c.png) - A _red_ commitment transaction, shown below, represents past, revoked commitment transactions. If Alice or Bob were to sign and broadcast one of these then the other party would be able to sweep all their funds. So these revoked transactions can effectively be considered invalid. - A _yellow_ commitment transaction represents the latest, valid set of commitment transactions. These are the commitment transactions that would be broadcast in a foirce close. - Finally, each side has what can be considered a "staging area" where changes to the commitment transaction can be proposed. Later on, either side can decide when it wants its counterparty to commit to the changes in their staging commitment transaction. Hint: if you like git, this is very much like a git workflow. Past commits are out of date, but they tell a nice story of what happened. Your latest committed changes represent the current state of your project. Any changes not yet committed are in staging. ![3-legend](https://cdn.satellite.earth/9a939ce1d3dcf7be15ca49970513505571a3355559620cd4d5f869dc017e9927.png) #### Adding a HTLC When either Alice or Bob want to send a payment, they would need to propose the inclusion of the HTLC to their channel peer. This is done with the `update_add_htlc` message: ![update_add_htlc](https://cdn.satellite.earth/b6cd4df7050cccda7eab624c24a9617602bf7059c745c2a1ba273aa212190f32.png) Here's a table for what each of the message fields mean: | Field | Description| |---|---| | channel_id | used to communicate which channel this change should take place on. | | id | an always-incrementing identifier for this propsed change from the sender. | | amount_msat | the amount that should be attached to the HTLC output. | | cltv_expiry | the block height that the HTLC should expire at. | | onion_routing_packet and blinding_point | data that the HTLC recipient will use to determine where next to send the payment. | There is a counterintuitive point to note here: the sender of this message does not yet put this update in their own staging area yet. This is because the update is still "pending on the receiver" because the sender has not yet received any acknowledgement from the receiver for the new update. Imagine this scenario: the sender sends an update and then immediately receives a `commitment_signed` message (more details on this later). In this case, there should be no ambiguity around which updates are included in the transaction being signed. Hopefully this will make more sense as we work through the example. ##### Step1: Alice -> Bob: update_add_htlc(A1) Let's say Alice sends Bob an `update_add_htlc`. Let's call this HTLC `A1` because it's the first one Alice(A) is sending to Bob. If Bob is happy with all the fields in the messag, then he adds the HTLC to his staging area commitment transaction nad Alice marks the HTLC as pending on Bob's side but does not yet add it to her own staging commitment transaction. ![add_A1](https://cdn.satellite.earth/bb7d88668bbd8d291638b6abbf600a8bfefc780ed6ea9fafdce7b5204683a73d.png) Note that neither side has actually committed to this HTLC yet so if Bob is a routing node for this payment, he should not yet send `update_add_HTLC` to the next hop on the route until `A1` has been _irrevocably_ committed. An HTLC addition or removal is only considered irrevocably committed once both parties in the channel have committed to the commitment transaction with or without it. This is not shown on the simplified diagram, but in reality Alice's main output in Bob's staging commitment transaction (i.e. the `to_remote` output) will now have the value of the added HTLC subtracted (along with the fees to cover the extra output). If the HTLC ends up succeeding then this amount will be added to Bob's output and if it ends up failing, then it will be re-added to Alice's output. ##### Step 2: Alice -> Bob: update_add_htlc(A2) Even though they have not committed to HTLC `A1` yet, this does not prevent them from adding more changes to the staging area. Alice is more than welcome to propose a new HTLC, `A2`, to Bob: ![add_A2](https://cdn.satellite.earth/38af194feda33cf0de580636d4b25555b503b46230fe32d762638c3b7fac723e.png) ##### Step 3: Bob -> Alice: update_add_htlc(B1) Likewise, Bob can sugges a change, `B1`, to Alice: ![add_B1](https://cdn.satellite.earth/6f97e8d11a0b86f3a83a336991298f24756bed0fcf89b3a70ac8fd17bc579f62.png) #### Committing to the current state At some point, one the peers will want to make sure that the other peer has committed to the latest set of changes and revoke the previous valid state. The is done by sending the `commitment_signed` message: ![commitment_signed](https://cdn.satellite.earth/8f3d3997cdd5aaf9497fdf48c7ea4ff2a5494ec722edd19f2b63d40731e7dc60.png) | Field | Description| |---|---| | channel_id | used to communicate which channel this change should take place on. | | signature | the sender's signature for the remote party's staging area commitment transaction. | | num_htlcs | the number of HTLCs that he sender expects to be on the remote commitment transaction. | | htlc_signatures | an array of num_htlcs signatures which are the sender's signatures for each of the second-level HTLC transactions that the remote party would need to broadcast if they were ever to force close the channel. | Hint: if you need a reminder about second-level HTLC transactions and why they are needed, check this [chapter]. ##### Step 4: Alice -> Bob: commitment_signed Let's say that Alice sends this message to Bob. Bob will now have all the required signatures from Alice to broadcast his staging area commitment transaction: ![commit_sign_1](https://cdn.satellite.earth/b389eab734e9c8242e17b6fff02a2572e1f58009817540f27dba2720a1652738.png) Notice that Alice knew that her signature would need to cover the `A1` and `A2` HTLCs. This is fine because the [underlying transport](https://github.com/lightning/bolts/blob/master/08-transport.md) is guaranteed to be reliable and ordered meaning that if Bob were to receive Alice's `commitment_signed` message then it means that he definitely received her `update_add_htlc` messages for `A1` and `A2`. Alice and Bob both know that the signature should not cover the `B1` HTLC since Alice has not sent an acknowledgement for it yet. Another thing to notice is that Bob now actually has two valid commitment transwactions since he has not yet revoked his previous state. However, he is incentivised to revoke his previous commitment transaction since the HTLCs on the new state are one of the folowing: - Payments to Bob himself meaning that he gains from committing to the new state. If he doesn't commit to the new state, then Alice won't either and so there would still exist a version of a commitment transaction that does not pay Bob his incoming funds. - Similarly to the point above, if Bob is routing a payment, then he is also incentivised to irrevocably commit the HTLC since he would earn routing fees if the payment succeeds. - Finally, if Bob is making the payment himself, then the first state would in fact be more profitable for him since he has less funds in the second state. However, the merchant that Bob is making the payment to won't release the goods being purchased unless funds are first received, which won't happen if Alice does not pass on the HTLC which she won't do unless it has been irrevocably committed to. So again, Bob is incentivised to revoke his previous state. ##### Step 5: Bob -> Alice: revoke_and_ack In response to Alice's `commitment_signed`, Bob sends the `revoke_and_ack` message: ![revoke_and_ack](https://cdn.satellite.earth/110b806997af5758d62cb36f3380bd6180044179bc31ab644f11c7f22f5a3d16.png) - The `per_commitment_secret` provides Alice with the information she needs to be able to spend any revocation path on Bob's previously valid state. See [revocation chapter] for more detail about how revocation works. - The `next_per_commitment_point` gives Alice the information she needs in order to derive the revocation pubkey that will be used in Bob's next commitment transaction. Once Alice receives this message, Bob's previous commitment transaction has successfully been revoked. Note that this message explicitly acknowledges the `commitment_signed` message sent by Alice and by extension, since message delivery is reliable and ordered, it also implicitly acknowledge the `update_add_htlc` messages that Alice sent for `A1` and `A2`. Alice can therefore finally add `A1` and `A2` to her staging area commitment transaction since they are no longer pending on Bob's side: ![8_revoke_1](https://cdn.satellite.earth/beb1adf052f2c8aabfc582023a2fc39d9f1a4d62b27a78d6164889bb86b6083f.png) Now we can clean up the diagram a little: ![9_clean_up_1](https://cdn.satellite.earth/5b68f0d46e5e4d2047516f7f061929ca9a3707f2297510e0c0f643c74ef1443a.png) From the last diagram, notice that Alice's and Bob's latest commitment transactions are actually out of sync. This is fine since none of the updates have been irrevocably committed yet. That might seem hard to believe since the commitment transactions look so different so let's walk through the consequences of either of these transactions ending up on-chain from the perspective of both sides. - From Alice's perspective: - if her commitment transaction were to be broadcast, she gets back her original `to_local` amount. - if Bob's commitment transaction were to be broadcast, then Alice's offered HTLCs (such as `A1` and `A2`) will be on-chain. For offered HTLCs, alice is sending sats _out_, meaning that her `to_local` would be lower. But, if Bob was a routing node for these HTLCs then he would not have forwarded them as they are not yet irrevocably committed meaning that he won't receive the pre-images required to claim these HTLCs and therefore Alice would be able to get her funds back via the timeout path. If Bob was the recipient of these HTLCs, then he would be able to produce the pre-image to claim the HTLCs, but then Alice would see the pre-image on-chain and would be able to claim the incoming HTLCs from their incoming channel and would thus have earned routing fees. - From Bob's perspective: - if Alice's commitment transaction is broadcast, Bob gets back his funds via the `to_remote` output. - if Bob had to force close via his commitment transaction, then HTLCs offered to him (like `A1` and `A2`) would be on-chain. If Bob was routing these HTLCs, then he would not have forwarded them on since they are not yet irrevocably committed. So he wouldn't be able to claim the success path but that is fine since the funds for these did not come out of his balance. If Bob waas the final destination for these, then he would be able to claim them via the success path. ##### Step 6: Alice -> Bob: update_add_htlc(A3) We want to be very clear about the point that the commitment transactions can remain out of sync indefinitely and that Bob does not need to send `commitment_signed` just because Alice did. So, for the sake of another example, let's say that Alice at this point sends yet another HTLC, `A3`, to Bob: ![10_add_A3](https://cdn.satellite.earth/44331e013b657fc70ae31c91c4c62fb586439278fa037a0ec06e7f98efadcc13.png) ##### Step 7: Bob -> Alice: commitment_signed Bob wants to irrevocably commit some of the HTLCs so that he can forward them on, so he finally sends Alice a `commitment_signed` of his own. This will include his signature for Alice's staging-area commitment transaction along with all the signatures required from him for the second-level HTLC outputs. ![11_commit_sign_2](https://cdn.satellite.earth/4da5f2c7fa2d65b5727c00548a6377cbc6dcc936452a4ac7fcce24ffd9da50d7.png) ##### Step 8: Alice -> Bob: revoke_and_ack Just like Bob did previously, Alice responds to the `commitment_signed` with a `revoke_and_ack` in order to revoke her previous state. This also serves as an acknowledgement to Bob that Alice has received and committed to `B1` and so Bob can now add `B1` to his staging area commitment transaction. ![12_revoke_2](https://cdn.satellite.earth/7738c53870ade1279a194a05b5983fecf7c803ac80775c9c29e004b0653d513d.png) Quick clean up... ![13_clean_up_2](https://cdn.satellite.earth/28d98986118ec103c7ac32227c7d30ef8d62dd29d1404bd274858aa21a76ac7a.png) At last, we have some irrevocably committed HTLCs! `A1` and `A2` have been irrevocably committed. `B1` and `A3`, however, have not since they have not yet been committed to by both parties. Let's quickly irrevocably commit `B1` and `A3` as well. ##### Step 8: Alice -> Bob: commitment_signed ![14_commit_sign_3](https://cdn.satellite.earth/6290211b31cd2fd24bceaefa2e305f638f51fffc4b7618553d671fcecee45802.png) ##### Step 9: Bob -> Alice: revoke_and_ack ![15_revoke_3](https://cdn.satellite.earth/396195b96958e4d03300c9fd456c10af3f5e584bf842b55c8e83c04ddd77220c.png) ##### Step 10: Bob -> Alice: commitment_signed ![16_commit_sign_4](https://cdn.satellite.earth/45d22f9c3a633df7719fb093dac6f7223f027000ee113f7050d96391c05667ba.png) ##### Step 11: Alice -> Bob: revoke_and_ack ![17_revoke_4](https://cdn.satellite.earth/1858c2a34cd2c6fa1985ecf1fc056c507048c9b7f872b46c82a4f9dac24832bd.png) Cleaned up: ![18_clean_up_4](https://cdn.satellite.earth/4f68e7ba933b29e6ca98006d3948dc1fb64b378e5cb51e2bb4fb0cf0d5d8ffce.png) Now all the HTLCs have been irrevocably committed! #### Removing HTLCs You probably get the idea of adding HTLCs now. But what about removing them? HTLCs are removed if a payment succeeds or if it fails. Note that HTLC removal messages can only be sent by the peer who did not send the original update_add_htlc and that HTLC's are only removable once they have been irrevocably committed to. Luckily for us, all the HTLCs have been irrevocably committed, so wee can start removing them now. ##### Step 12: Bob -> Alice: update_fulfill_htlc(A2) In the best case scenario, a HTLC is removed because it is being fulfilled meaning that its pre-image is being passed back. This is done with the `update_fulfilled_htlc` message which looks like this: ![update_fulfill_htlc](https://cdn.satellite.earth/3c2e9c97538d2e8bb0bb4e11ba3609213c28941416bef054dc065490deee72e3.png) In our example, Bob sends Alice the `update_fulfill_htlc` message for HTLC `A2`. This also demonstrates that HTLCs don't have to be removed in the same order they were added. ![20_fulfill_A2](https://cdn.satellite.earth/8b56cbca75a4352728e5b50c72d3eeec5187344fc38a965e65aa8b2dfb9e3821.png) Notice that just like the updates for adding an HTLC, updates for removing an HTLC will also initially be pending on the receiver side until they have been acknowledged by a `revoke_and_ack`. So in our case, Alice removes `A2` from her staging area transaction when she receives the `update_fulfill_htlc` (and allocates the HTLC amount to Bob's output) but Bob does not yet remove the HTLC from his staging area transaction. Unlike other update messages, there is no need to wait for an HTLC removal to be irrevocably committed if you receive the pre-image for it. After all, you can immediately send the pre-image upstream in order to claim any HTLCs there. ##### Step 13: Bob -> Alice: update_fail_htlc(A1) HTLCs can also be removed due to payment failures such as HTLCs timing out or if there was some sort of routing failure such as a specific channel on the path no longer existing, a hop's fee requirements not being met, a link not having sufficient balance, etc. Such failures are communicated with the `update_fail_htlc` message: ![update_fail_htlc](https://cdn.satellite.earth/225b621432aab29d35d12e7419673d0dfd34ba8c8b50d8fda40988863b8d6f2b.png) The reason field is an encrypted blob for the sender of the payme3nt in order to inform them of the failure reason. After Bob sends Alice the `update_fail_htlc` message for `A1`, the state looks as follows: ![21_fail_A1](https://cdn.satellite.earth/bd568b170a857ae4d795f51f0bc1df39f39bd113e8338928a7948076e0ee70b7.png) ##### Step 14: Bob -> Alice: update_fail_malformed_htlc(A3) The final message that can be used to remove an HTLC is the `update_fail_malformed_htlc` message: ![update_fail_malformed_htlc](https://cdn.satellite.earth/c6ac201b76a0fe3e85e0e109a9571fb66f69deb270da270741b664d91ce5f2a8.png) This message is sent if any hop on the payment route was unable to parse the `onion_routing_packet` it received in `update_add_htlc`. If Bob sends Alice this message for `A3`, then the state now looks like this: ![22_fail_A3](https://cdn.satellite.earth/6c826e1aedefc9e60fd460c75db26eed86871195942a3f9553eb3f3bed1c3a6e.png) ##### Step 15: Alice -> Bob: update_fulfill_htlc(B1) Alice also initiates the removal of `B1` by sending an `update_fulfill_htlc` to Bob. ![23_fulfill_B1](https://cdn.satellite.earth/60596bf1d92ea2444a28f0aef77d5076319d8b50510a7e1fe649ac13539387a2.png) Let's now clean up the HTLC removals by irrevocably committing them. This will require a few `commitment_signed` - `revoke_and_ack` flows: ##### Step 16: Bob -> Alice: commitment_signed ![24_commit_sign_5](https://cdn.satellite.earth/2238d0cf78e60444cca3705b9e626fa16c71fe0a54df13e8ca5812ad8e784cf9.png) ##### Step 17: Alice -> Bob: revoke_and_ack ![25_revoke_5](https://cdn.satellite.earth/9fe8849186b8839936c1784022f739c50c48a2b79e33a94fb9e781f7c5ed5e98.png) ##### Step 18: Alice -> Bob: commitment_signed ![26_commit_sign_6](https://cdn.satellite.earth/a6a2f1a22a25dcacf63dba076fa96e52f32823fdcb0034cccae1586fba0bbd4a.png) ##### Step 19: Bob -> Alice: revoke_and_ack ![27_revoke_6](https://cdn.satellite.earth/de07ede026ec02bede1d15fca68e1ac139bfb361a2264d6881499a5de13ead0b.png) ##### Step 20: Bob -> Alice: commitment_signed ![28_commit_sign_7](https://cdn.satellite.earth/51065296b250c77282b6229698bd8ae037b3f98556677a9e46b0ff5fafc45106.png) ##### Step 21: Alice -> Bob: revoke_and_ack ![29_revoke_7](https://cdn.satellite.earth/6944334d80c8afc4431e1674b0461d430a3b3764567ff3b6bd206e249aff878b.png) The two valid states now look nice and clean again: ![30_clean_up_7](https://cdn.satellite.earth/acae49b81538acee7368fedbff4123b825c9818075efc75b62e137262c931ca9.png) ### Updating fees There is one more `update_*` mesasge that we need to cover and that is the `update_fee` message. This message is used to update the fee-rate that the peers should use when constructing their commitment transactions. The original fee-rate is decided in the open-channel flow but if the average mempool fee-rate increases, the channel funder might decide to update the fee of the commitment transactions so that they have a better chance of getting confirmed in a timely manner in a force close situation. It could also be that when the channel was opened, a very high fee-rate was chosen and perhaps a lower fee-rate would be desired. This message follows similar rules to other `update_*` messages in that it must also be irrevocably committed before it takes effect. The only extra rule that applies to this message is that only the channel funder may send this message: ![update_fee](https://cdn.satellite.earth/a5ec70daf2c18ba4eee6a811eec28c06767be959ed12d2fa0597b2b84d3e9447.png) Note that with anchor channels, the need to use the `update_fee` messaging is becoming less and less since nodes will be able to use CPFP on the force-close transaction if required. ### Message retransmission Something that you may have picked up on in the add/remove HTLC flow is that explicit acknowledgements for the `update_*` messages is delayed until the `commitment_signed`/`revoke_and_ack` exchange. That is ok most of the time since we assume that the underlying transport between the two nodes (see [BOLT8](https://github.com/lightning/bolts/blob/master/08-transport.md)) is ordered and reliable. However, if for some reason the connection between the two nodes needs to be re-established for some reason, there will be doubt regarding whether our peer has received the last message that we sent. This is where the `channel_reestablish` message comes in. Upon reconnection, before continuing with the noirmal operation flow, the peers will exchange this message to make sure they are on the same page and to determine which messages they possibly need to re-send to their peer. ![channel_reestablish](https://cdn.satellite.earth/230627662489aa6e20b342cff914a7d5ef68ad5b15abf81567beb005e0812297.png) Each peer in the channel has their version of the commitment transaction and the two commitment transactions can be updated independently, meaning that the number of times that one side's commitment transaction state has been updated (through the `commitment_signed` and `revoke_and_ack` flow) could be completely different to that of the other side. The `next_commitment_number` field in the `channel_reestablish` message allows us to communicate with our peer the next `commitment_signed` that we expect to receive from them. This way, they will know if we have perhaps missed a `commitment_signed` from them that they previously sent before the disconnection. In other words, `next_commitment_number` tells the remote peer what we see our latest committed state to be. Similarly, `next_revocation_number` is the commitment number of the next `revoke_and_ack` that we expect to receive. In other words, this indicates to our peer which commitment number we think is their latest one. The `your_last_per_commitment_secret` is the last per-commitment secret received from the remote peer which will give the remote peer an idea of the state it has definitely revoked. `my_current_per_commitment_point` is the commitment point of the local party on its last commitment transaction signed by the remote peer (in other words, the commitment transaction that has not yet been revoked). There are lots of checks that a node should do when receiving a `channel_reestablish` in order to make sure that all necessary updates are re-sent so that the channel can continue working as normal. There are also some checks that ensure that nodes are not tricked into revoking a state that should not yet be revoked or tricked into broadcasting a state that _has_ been revoked. If you are interested in the details around these checks, read [BOLT2](https://github.com/lightning/bolts/blob/master/02-peer-protocol.md). Note that when a connection re-establish happens, both sides must remove any uncommitted updates from their staging area. If we revisit the git analogy, they should use git stash when a reconnection occurs. This means that both sides will need to retransmit any `update_*` messages that were not yet committed on the other side's commitment transaction.