Rene Pickhardt is an independent researcher working on improving payments on the Lightning Network.
In our talk, Rene explained his experience doing independent Bitcoin research, how multi-part payments and zero base fees can improve Lightning payments, and some of the limitations of the Lightning Network today.
→ Rene’s Website: https://ln.rene-pickhardt.de/
→ ZEBEDEE: https://zbd.gg/
At the end of every show, I answer any questions listeners send in over the Lightning Network.
To ask a question, send a message, or to support the show, download Fountain from the App Store and load your wallet with a few sats.
→ Fountain: https://www.fountain.fm/
→ More Episodes: https://play.fountain.fm/show/P6XXuSPg6f2rj4ECB0fT
→ Lightning Address: ⚡email@example.com
→ Twitter: https://twitter.com/kerooke
→ Blog: https://www.kevinrooke.com/blog
00:00 - Intro
02:21 - Rene Pickhardt Intro
10:40 - Rene’s Experience Writing Mastering The Lightning Network
15:41 - Rene’s Experience as an Independent Researcher
27:41 - Multi-Part Payments
47:51 - Reliability, Fees, and Latency on Lightning
54:15 - Zero Base Fees
1:09:57 - Real-World Parallels for Lightning Network Architecture
1:16:41 - Limitations of the Lightning Network
1:21:01 - The Lightning Round
Rene Pickhardt Timestamp: 0:00
There was not really a reason to pay anybody, right? There were no stores accepting anything. And then lnbic came and spun up several nodes and basically provided inbound liquidity to random nodes. I can basically, completely on my own, decide what I want to work on, what I think is important for the network to work on, and do it in my own way and own pace, so to speak. I was actually recently wondering if I put my optimization criteria to become as rich as I wanted to be, how much would I earn in this space if I would just, like, solely focus on making money? I mean, this would be a fun experiment to some degree, but I decided for other things with my method. If I have a Lightning Node that has a Bitcoin on it and somebody else has a Bitcoin of inbound liquidity, probably I can send it over the network. If we don't provide basic reliability on a protocol level on the long term, we're going to be in a very messy situation. More liquidity is not necessarily a good thing. It may be right, but it depends. It's really complicated.
Kevin Rooke Timestamp: 1:09
Renee Picard is an independent researcher working on improving Lightning network payments. In our conversation, Renee discussed some of the pros and cons and his overall experience being an independent researcher in the Bitcoin ecosystem. We discussed how multipart payments and zero base fees can contribute to improving payments on the Lightning network, and we also discussed some of the limitations and some of the challenges of using the Lightning network today. I've also added Renee to today's show splits. So if you guys enjoy this show, if you learn something new and you want to support it, the best way you can do that is by sending in sats. All the SATS that you send in over the Lightning network will be split 50 50 between myself and Renee, and we'll both be able to read your comments and questions. Quick shout out before we get into the episode. Today's show is sponsored by Voltage. Voltage is the industry standard and next generation provider of Lightning network infrastructure. Today's show is also sponsored by Zebedee. That's Zebedee, and Zebedee is your portal into the world of Bitcoin gaming. We'll have more from Voltage and Zebedee later in the show. Renee, welcome to the show. Thank you for coming on today, and I've got a lot of questions for you. You're an independent researcher working on Lightning, and you're in a sphere of the Lightning network that I'm not super familiar with. So I got a lot of questions. I'm sure the audience is going to enjoy this one. Can we maybe start off with your background in Lightning and maybe the moment you first discovered the Lightning network and the moment you first recognized that it was an important technology worth studying?
Rene Pickhardt Timestamp: 2:49
Yeah, sure. Hi, Kevin. Thanks for inviting me to the show. I hope that we can make a great show for your audience. So I basically discovered the Lightning Network in early 2018 when Christian Decker published a blog post on the Blockstream block, where they basically said, hey, we have an implementation now that is reckless on mainnet. And a friend sent this to me and said, hey, it seems like blockchain systems might scale after all, right? So my background is I'm a mathematician and I worked in data science. So my main topics were statistical analysis, machine learning, but also scaling of web architectures, and scaling of software architectures in general. So basically, friends said, Hey, look, this thing might work, and I started reading the blog article. I spun up my first Bitcoin note ever, which was a very frustrating moment because it took like, almost a week for initial block download. Then I couldn't really start payment channels because how would I fund this? On chain fees were really high. But I read the Lightning white paper and I literally got nerd sniped by it. Right? So I found this really intriguing solution, and basically in my spare time, I started reading more and more. So I think I wrote the German Wikipedia article. I posted some really dumb questions to the mailing list. And then in June of 2018, I saw that there was the Lightning hectare organized by Fulham in Berlin. And I'm not located in Berlin, but I had an appointment on that weekend in Berlin anyway, so I was like, Hey, I could give it a try and go there. So I went there, and it was basically this life changing moment for me, because it was the first time I had ever contact with bitcoiners. And like, there were a lot of bitcoin, obviously, in this event. And I realized that over the past couple of months, I actually have learned quite a bit about the Lightning Network. Obviously not as much as I know nowadays, but it was certainly enough to have qualified discussions with some of the developers. And basically from that day on, I started to work on Lightning on a daily basis.
Kevin Rooke Timestamp: 5:06
Right. What was the discussion like in 2018? What was the vibe of that conference? What did you have in mind for where the Lightning Network was headed then? And how has that evolved in the last four years?
Rene Pickhardt Timestamp: 5:23
So, in the discussion, I think a lot of people from Lightning Labs were there. And I remember one of the first things that I discovered on the Lightning Network, and I didn't understand that this was not part of the protocol at this point was the so called Autopilot of Lightning L, and the Autopilot is a feature in LND that automatically would open channels for you. And when I looked at the code, I had the feeling that this was really poorly implemented. And I think it's the very first issue I ever opened in a repository when basically saying, hey, you're doing this, like, Barabbasi Albert model to make recommendations for channels. Is that really reasonable? So I remember in this Lightning hectare, it was like a barkham kind of style. So I had a session where I was saying, hey, can we discuss different types of strategies? Boy, little did I know that this problem is actually one of the larger challenges on Lightning network. I mean, nowadays if you talk about all the routing notes, the main challenge they have is like, where do they want to provide the liquidity and why? And I think to some degree it's still a very unsolved question. But I already had the feeling that this question is related to routing and reliability. So it was more like a kind of like common sense computer science feeling that I had rather than a deep understanding of how Lightning works and how behavior of people on the Lightning network is. But yeah, there was a lot of discussion around this, obviously, because I opened the Barkham session. And the other thing was, I think the Rusty Blitz project was kind of born there. So there was Ruthul who brought all this little Raspberry Pis and was like, hey, let's have a hack table and try to install a Lightning Node there. So I think the Git repository wasn't live yet at this point, but it was kind of like born. Like people were like playing around with this idea and it was really funny. Like almost nobody had a Lightning Node running there. Like I had Lightning node on my machine running and I remember that I was still like syncing some blockchain data while I was arriving in Berlin, right, because my note was offline for some time. So when I was going to the WiFi, I was basically syncing the rest of the blockchain like the last half percent or so so that I could have my Lightning load running and do stuff.
Kevin Rooke Timestamp: 7:41
Yeah. What's the most surprising thing then that's happened since then? Like having gone from that stage where basically no one was even running a Lightning node to now having this flourishing ecosystem of apps, like we have a country that has taken the Lightning Network and adopted it. What's been the most surprising event to you?
Rene Pickhardt Timestamp: 8:03
A lot of the really surprising things were actually really going on in the early days. And when I say early days, I'm talking about my early days, right? I mean, obviously people have been developing for this for already three years and I didn't know that. So I can't talk about those early days, but I certainly remember at some point in time LNBig showing up and just dropping 200 bitcoins of liquidity to the Lightning network. So what has happened before is like basically people testing this with really smallish channels. Routing almost always failed because the graph was just not very well connected and most of the channels were one sided. There was not really a reason to pay anybody. There were no stores accepting anything. And then LNBig came and spun up several notes and basically provided inbound liquidity to random notes. And as soon as somebody would also open the channel with Lmbic, then you would almost certainly be able to route to somebody else. So this was certainly a huge game changer in this phase. And then the other thing that I remember, I don't know if you remember this was the Lightning torch by Hodlnaut. So that was actually a fun one. Like in my Twitter, I was like very active in this, like, Lightning focused groups. And at one point in time I saw people sending invoices to each other and being like, hey, let's add 10,000 satoshi at a time. And I was literally going home, talking to my fiancee at that time, saying like, hey, there's this weird thing happening. This is so dumb, right? And it was really annoying me for one or two days, like, stuffing my Twitter timeline. But after two days, I think I grasped the severity of this and I was like, I want to do this too. And I basically think somebody is like, hey, send the torch to me. And then somebody already was like, hey, I want to have it next. So I sent it to Stefan, with whom I also wrote the paper about multi part payments. And at that time I was in Germany in a small magazine trying to see if I wanted to do reporting about Lightning. And I was actually the first person writing a media article about the Lightning torch thing. So even though I ignored it in the very beginning, I was still very early. And then at some point in time, jack obviously got to Jack Dorsey, and then it just exploded. Like, everybody wanted all those super famous people had the torch. Like, it was literally impossible to get it back. That was kind of a surprising thing because I think it was not really planned. I think Hodlnaut was just like playing around and messing around and just got viral.
Kevin Rooke Timestamp: 10:33
Right? Yeah, it was just like a spontaneous community thing that just developed.
Rene Pickhardt Timestamp: 10:38
Kevin Rooke Timestamp: 10:39
Okay, I want to get into routing in a minute, but first I want to talk a bit about the book that you're a co author on, mastering the Lightning network. So this is one of the foundational there you go. Video viewers can see that this is one of the foundational books of the Lightning network. I enjoyed the parts that I could get through. It's a very technical book, but there were early chapters where I felt like I could understand some of the discussion there. But I'd love to learn more about what you picked up in that process of writing it and what some of those things you learned about the Lightning network were.
Rene Pickhardt Timestamp: 11:23
Yeah, that's a good question. So maybe I can go one step back and explain how this book actually came to happen. Sure. So as I said in 2018, I joined the space, and obviously, I was still learning a lot of things. And I have been working in education basically my entire life. And for me, it was always the case that when I can explain something to somebody, I understand this very well. So I started in 2018 to do, like, technical YouTube videos about the Lightning Network. But I understood that a lot of people do not want to watch a technical YouTube video, right? They like to watch a YouTube video that is entertaining, or they like to watch a YouTube video that is like a podcast. But a technical video is not necessarily the thing, especially developers. So between 2018 and 2019, there was the Chaos Communication Congress in Leipzig, which is like one of the big hacker get togethers in Germany. Like, 20,000 people come and they have, like, a Bitcoin table there. I was talking to a few people and I realized, I think I should write a book about the Lightning Network. But it's missing. Like, a lot of people were coming to Stack Exchange to Taylor Grump channels, asking really basic questions all the time, over and over again. And I had the feeling, I want to write a book about the Lightning Network. So what I did is I sat down and I have another book I can show to the people who are in the audience. It looked like this, which was like the Lightning Network book by the Bitcoin community. And I basically announced that I would want to write the Lightning Network book in 2019. And I opened a fundraiser, and I hoped at that time that I would be able to collect 21 bitcoin, which, if we remember back the time, was roughly the equivalent of, I think, $75,000. There was a lot of excitement going on. I was able to collect 1.2 bitcoin, basically, which is not quite the same, but I was very obnoxious at that time. I was, like, literally starting to write the book. And then Andreas reached out and he was like, hey, Renee, why don't we do this together? He said that he was already planning this. He already talked with O'Reilly. He had a roast beef on board who also wanted to support this effort. And then we said, Yeah, well, we could do this with three people together. And that being said, the things that I learned most about were less technical, but more of how to collaborate with three people that have a very, very different perspective.
Kevin Rooke Timestamp: 13:57
Right now, if you were to pick out all the topics covered in the book, and if you could get up in front of all bitcoiners today and share one topic or explain one topic, maybe something that's often misunderstood in the Lightning Network, what would that topic be? If you could pick one from the book?
Rene Pickhardt Timestamp: 14:19
I think in general, we have this I think it's chapter three, where we basically talk about how the Lightning Network works which is kind of like an overview of all the components that I included in this, like the payment channels, the routing. And I think there are still a lot of misconceptions out there. So I would probably pick that and say to people, hey, read this. We try to make chapter three very accessible in the sense of even if you are not a programmer, you should probably be able to get your head around. I mean, if you understand bitcoin, it certainly helps. So that's probably one point. The other thing that Andreas at some point in time told me is like, my most valuable contribution to the book is in the beginning of part two, we have this diagram, this layered architecture of how the protocol works. I mean, what is true is that we had a lot of discussions of how to organize the book and where to put one chapter, and as soon as we had this diagram, basically part two of the book wrote itself. So this diagram certainly helped us a lot to understand how to structure stuff because the protocol itself is quite like there's a lot of things depending on each other. And this diagram kind of like put a little bit of order in it and clean certain things up.
Kevin Rooke Timestamp: 15:40
Interesting. Okay, so now in writing this book, and even beforehand, it seems like you had a passion for doing things independently, having started your own book almost on your own and then coming on board with the other two co authors, and now you're an independent researcher. Can you talk to me about that experience as an independent researcher? What are some of the pros and cons of trying to create and do great research without being under a corporation?
Rene Pickhardt Timestamp: 16:14
Yeah, so I guess the biggest advantage is it gives me a lot of freedom. I can basically, completely on my own, decide what I want to work on, what I think is important for the network to work on and do it in my own way and own pace, so to speak. Right. If you want to do corporate research or even university research, you often have to write proposals. You have to convince people that this may or may not be interesting, and especially the donors might give you feedback and say, hey, we would like to fund you, but if you would do it a little bit differently, like, if you would go in this direction. Right. So in that sense, I think the biggest advantage is like the vast degree of freedom that you get. This, of course, comes to a certain price, and that is especially the price that I already touched is the funding. But for a long time I was heavily underfunded. This has changed quite a bit with the results that we presented last year. Since then, my funding situation has improved quite drastically. But before that, for the first three years, I was basically always looking out for ways of like, getting reimbursed for the stuff that I would be doing. And the biggest problem was companies, of course, were seeking out and asking my advice for consulting. But the first thing that usually happens is, like, here's your NDA. And I'm like, I cannot operate under this, right? I don't want to operate under this. Right. I think there are certain things in this space where it's important that everybody has a shared knowledge and a shared common sense of what is going on. So even if you go to certain open source companies, you cannot talk about everything that you're doing publicly. Everything is very political because there are business models involved, and I'm just not bound by these things.
Kevin Rooke Timestamp: 17:59
Right? And so now, if someone is considering taking the path of being an independent researcher, now that you've gone through this experience and you've kind of seen the struggles of getting funded and then now in a position where you are funded and you feel like you can commit your time to doing your research, what's the playbook here? What do you suggest when other people say, hey, I want to be an independent researcher as well?
Rene Pickhardt Timestamp: 18:26
Yeah. So what I did, basically, is the way how I secured funding was by finding a position at a university, right? So you could already argue, well, you're not independent anymore because you have a tie with the university. But I was lucky to be, like, a good negotiator there. So I was basically negotiating that I can decide my topics and projects on my own. And I made it very clear from day on if that is not the case, like, if this is just a promise, I will be gone and find something else. And I think the leverage that I had is that it was always clear that I could find a job outside of the university. And I think this is also one of the problems. Right? I mean, of course, it's very easy to be an independent researcher if you're just rich. I am not rich. Right? I had to make this deal and basically make this bluff in the university of saying I do have different options. Right? So, yeah, you need to find a certain space. Also, what certainly helped me is when I was on this first Lightning hecte, one of the Bitcoin OGS that was there was basically looking at me like, why do I not know you? Why have you never been here? You seem to know quite a bit while I'm doing other stuff. And the person was like, what are you doing? I'm like, I'm a data science consultant. He's like, Why don't you work on Lightning? I'm like, I do in my free time. And she's like, Yeah, well, I could give you a monthly stipend, and then you could work more on Lightning. Right? So I think that was a very fortunate thing because this person was not very technical, and he just basically saw this as a bootstrapping kind of thing. So in my first year I was basically being funded in this way. The statement was not like really high, but the deal was basically, hey, you do your data science consulting but don't acquire customers too heavily. Like whenever you have free time, just work on Lightning because your basic costs are being covered. Right? And I was just willing to take that. Right? So a large part of being an independent researcher is really the stamina that comes with it and the deep conviction that you want to be independent and that you think this is important.
Kevin Rooke Timestamp: 20:33
What do you think about the state of independent research in the Bitcoin community? Do you think we're doing a good job of promoting that and getting people on board and able to the point where they can actually create research themselves? Or is that something where the Bitcoin community should focus on improving?
Rene Pickhardt Timestamp: 20:54
I mean, I might be not that independent in answering this question, right, so if I speak for my lobby, I would say we could certainly work on improving ways of doing this. Right? I mean, there are obviously funding situations and opportunities there. There are the universities which are great, like on ramp for people to become independent. But often in universities you see people being very bound by university constraints and I mean, I would argue independence and freedom is something that just comes at a price. Like it's not for free. You cannot just magically create this. It's something that you have to deliberately decide that you want to do and then you have to find your work to fight for your freedom. And if you're willing to do this, I think there are ample opportunities to do that. And if you're not willing to do this, well, then you find other ways. Right? I think life in general is to some degree an optimization problem and it really depends on what you're optimizing for. And me personally, I always liked to optimize for my independence and for my freedom. Maybe that's one of the things that were so attractive in Bitcoin in general for me.
Kevin Rooke Timestamp: 22:03
Right, yeah. It seems like in the last couple of years I've seen one of the talking points of crypto enthusiasts is that everyone can get funding really easily because they're spinning up a bunch of tokens. They're not necessarily independent but they are able to earn large salary for creating stuff on other blockchains because they have the component of the new token that they can create and mint and kind of own. I wonder whether or not you think that is. How does Bitcoin overcome that from that advantage where we can't necessarily say, hey, we're going to give you this brand new token because we don't have it. And that's a great thing, but it's in the short run something that maybe it's a good thing because it doesn't bring in the people who are looking for a short term gain. I guess they kind of go towards the crypto projects that will offer that. And the Bitcoiners may have to kind of maybe it does attract the people with long term mindsets. I would love to know what you think about that.
Rene Pickhardt Timestamp: 23:18
I mean, I kind of agree with what you say, but obviously privacy is a thing, right? So I don't know who is behind all those token altcoin projects. I mean, I would assume that potentially some Bitcoin as are also behind them and try to refine some of the stuff that they are doing. I know that there was this point in 2019 where I was really frustrated about the funding situation that I had, but I was actually considering to do an April fools of creating dogecoin cash, right? And be like, hey, I am the real Jack Palmer. And then I would fork this again and say, hey, this is Dogecoin, Satoshi's vision or Jack Palmer's vision or whatever, right? You could easily do this if you're a developer, if you understand these technologies, right? I mean, if you do this as an April fools joke, it's even safe, right? You'd be like, hey, it was just a joke. But if it pulls off, you'd be like, hey, I'm into that many coins, right? It's ridiculously easy to do that. I personally decided against this, right? But again, I just said before, I think life is an optimization problem. So I was actually recently wondering if I put my optimization criteria to become as rich as I wanted to be, how much would I earn in this space if I would just solely focus on making money, right? I mean, this would be a fun experiment to some degree, but I decided for other things, right?
Kevin Rooke Timestamp: 24:45
Right, yeah. And it seems like that does attract a lot of Bitcoiners. That is not necessarily all about the money. I think to an extent we have to get to the point where we can fund people to work on Bitcoin. I think for a long time, as you mentioned, in 2019, there was genuine concern around in the community about that. Do you think that companies building on Bitcoin and some of the corporate sponsors, do you think they have a role in bringing on people through grants? I know there's a few companies in the space that are doing that. It's not necessarily independent entirely, but it brings on people who are dedicated to building on Bitcoin. Do you think that's an important role in the community?
Rene Pickhardt Timestamp: 25:32
Well, for example, I recently accepted a grant from Big Max, right? So, I mean, you could already question my independence, right? But what I can tell you is that Bitmix is not my own source of funding, right? So there's a certain amount of decentralization here and with the companies being in the space and I write this on my website, there's obviously always the risk that I mentioned before that even in university research, like whoever brings the funds for research or for work might have a certain say in what the work is about. Right. And in general, it's very hard to onboard people. I think there is some more Bitcoin project where they try very heavily to onboard people and to attract talent. But I also can say from my own experience, it's actually very frustrating, especially if you come with new ideas. It's much easier if you go to the GitHub repository and look at open issues and you try to solve them. Then what I did is basically looking at the problem and saying, I think we have this problem, I think we have that problem, and I think this would be interesting. Right. So you're being very, to some degree, creative, but I was talking a language and bringing problems that cryptographers were not necessarily interested in. And I have the feeling in the first years it was actually a lot of grinding to convince people that what I had to say and bring to the space was actually valuable to them. Again, with the results and with the time, this has changed, which is a good thing, but it's a process of long grinding and not many people are willing to do this or are in the position to do this. Right? I mean, if you are, let's say, a student in the United States so, for example, I come from Germany. For me, studying was free. I don't finish college with a huge sum of student debt. But if you're in the United States and you have like a quarter million of student debt, there is not the chance of like, hey, let's continue my student lifestyle for another three years and try to see where this ends. Right? I mean, you need to basically pay back your loan, right? So it's really hard for people to do that, actually.
Kevin Rooke Timestamp: 27:38
Right now I want to discuss some of the work you are doing because you wrote this paper, I believe, last year on multipart payments. And maybe we can start the conversation off with a discussion about what multi part payments are and why they're important.
Rene Pickhardt Timestamp: 27:56
Yes. So I basically published two papers last year, the one I actually wrote the year before, but I only published it last year, which was what I always say, a so called probabilistic pathfinding paper. Right. The title is a little bit different. It's like security and privacy on the Lightning network with uncertain channel balances. But the key here is whenever I make a payment, I don't know how the liquidity is distributed in the channels that I'm going to use. Right. So, Kevin, if we do have a channel, I can easily send money to you on our channel if I know that I have the money. But if you have another channel, let's say with Andreas, and I want to send money to Andreas over your channel, I mean, I can offer you an HTLC and say, hey, can you please forward 1000 or 100,000 satoshi on your channel to Andreas? But what could happen is that you are telling me is I just don't have that money on the channel with Andreas, but then you're going to fail your onion and the payment fails. Right. So what we did in this first paper is we express the likelihood for payments to settle on the Lightning network. And when you do all the math theory, what you will very easily learn is, and it's very intuitively to many people, is the smaller the amount you send, the higher the likelihood is that it will settle. Right. And this is a huge motivation for people to say, hey, instead of sending the entire amount at once, let's just split it into several parts and then send it across different routes through the network because each of the shards that we have has a higher chance of being successful. What people have been overlooking in the past, or kind of like not taking too seriously, was the observation that the more channels are involved in a payment, the likelihood of success also falls exponentially. Right. So there's this sweet spot if you have a certain payment size to a certain amount, like if you don't split it, your failure rate will be high. If you split it a little bit, the failure rate decreases. But if you split it too heavily, it increases again. Right. So this entire thing is an optimization problem. Like there is this sweet spot of how you want to split it. Most implementations did initially a very, I would say poor job in splitting payments. They would just basically say, hey, let's cut it in half or let's cut it into ten chunks of equal size. Right. But if you want to split it optimally, you might have to send like one chunk that is a little bit larger over, like larger channels and one chunk that is a little bit smaller. Right. And this entire splitting problem is kind of a weird mathematical problem. And this is something that I was kind of heavily working on at the beginning of last year.
Kevin Rooke Timestamp: 30:40
You're trying to figure out basically like, which if you and I are trying to send a payment back and forth and there's tens or hundreds of paths that we could potentially take around the network. The idea here is you're trying to figure out which ones are the optimal paths to take and should it be split up across many paths or should it just be confined to a few and figure out exactly what the optimal route is. Is that correct?
Rene Pickhardt Timestamp: 31:06
Yes, exactly. And I have to say, optimality obviously can mean different things, right. So for example, for you, optimality could mean the lowest fees. For me, at that time, optimality meant highest reliability, which is equivalent to highest chance for the payment to succeed. So I was maximizing the probability or reducing the uncertainty cost that is attached to not knowing how the liquidity is being distributed. Of course, you can also combine these two goals where you say I want to find something that is optimal with respect to my objective. That is minimizing the uncertainty and minimizing the fees. Or you could also include something like latency, right? So there's obviously a choice to be made, right? But I was literally studying this optimization problem. And while studying this optimization problem, as I said, I was a mathematician. I was writing this down very formally. I already had it written down, but I couldn't solve it. And I was basically reinventing this entire theory of minimum cost flows that was starting to be invented like 60 years around at MIT. And then Stefan Risk came around, a good friend of mine who understood the first paper about probabilistic pathfinding. And I was like, Stefan, I have this thing, I think this is working, I think this is optimal. But I cannot prove this and I don't want to publish this if I cannot prove this. Can you give me some feedback? And he's looking at what I'm doing and he's like, renee, can you please stop being so dumb? I'm like what do you mean? And he's like I'm paraphrasing him, right? He might not have said this literally, but he's like the stuff that you're doing, this must have been solved already. I'm like, I know, I tried to search for it in the literature, I didn't find anything. It turned out I used the wrong terms. And after a week or so, Stefan came back to me and he was like hey, there's this thing called minimum cost flows. You should really look at this. So then we were basically like one month just reading a 1000 pages book about minimum cost flows and we're realizing everything that I was trying to reinvent and prove was already solved by the people at MIT and some other universities. So then we could basically apply this entire theory of logistics and this minimum cost flow idea.
Kevin Rooke Timestamp: 33:16
This was outside the Lightning Network, right?
Rene Pickhardt Timestamp: 33:18
Totally, totally separate fields. This is in a field that is called Operation research. The older you get, the harder it is to be intimidated by certain things. But last week I was at MIT and before I went to the website of the Operation Research Council that's the department that is doing this. And I was just looking at this and I was so intimidated. It's unbelievable. I mean, it's hardly known to people outside of math, but the main cost for problem is I would say probably one of the mostly spread optimization mechanisms or solvers that we are using. So for example, the entire Covid response that they did in the United States, they have like was basically consulted by the MIT department there because there's a lot of logistics involved. Like how do you move the vaccines, like how do you get this to certain spots, right? I think this entire field operation research came originally from military, where you also ask yourself, how do you do the logistics of getting certain things at certain places, right. So you can already feel it's very similar to Lightning. Like, how do you get your satoshis from your node to another node, right. And you have a certain cost attached to each channel. And the cost, again, might be the fees, it might be the uncertainty, like, whatever you model into this. Our observation in this paper was basically like, initially I thought I would solve a tiny problem of, like, splitting this optimally. Right. But what turned out is that splitting this optimally leads actually to a huge improvement in the amounts that we can send over the Lightning network. So I think last year everybody agreed, like, yeah, if you want to send 100,000 satoshis or 200,000 satoshi, that usually works. But as soon as the amounts get larger, like, you have really high failure rates. With my method, if I have a Lightning Node that has a Bitcoin on it and somebody else has a Bitcoin of inbound liquidity, probably I can send it over the network. Wow. But I might not want to do it because the fees that I pay on Lightning are higher than an onshore transaction. But if we use Lightning more and more, and on chain transactions might also rise because more channels are being opened and closed maybe one bitcoin, and paying the fees on Lightning might be feasible. Right. So this is a total game changer.
Kevin Rooke Timestamp: 35:30
Yes. And this is something I've actually heard from a number of people in the space that in the last couple of years, the reliability of payments has dramatically improved. And do you think that the reason for this is because of the multi part payments? Personally, I've only been in this space for about a year or so, and I've noticed a little bit of an improvement in sending payments back in 2021. Whenever I try and send payments, a lot of my first ones failed. And I was getting frustrated with trying to figure out how the node worked and why I couldn't do certain transactions. And now it seems like I've got a lot less concern about that. And I go into a payment doing roughly the same amount, hundreds of thousands of sats, and I feel relatively confident now that it's going through, I guess. To what would you attribute that increase in reliability? Are there many factors at play?
Rene Pickhardt Timestamp: 36:27
So, first of all, try a million to my note and you will see that you run into problems. Or try 10 million to a note. Right. I mean, you can go to my website right now and fetch yourself an invoice and try it. I would argue it's safe to do because you're probably never going to, like, settle that invoice. And if you do, I mean, you show me a pretty image, I might reimburse you. You could try that, right? If you trusted me that I would reimburse you. But now comes my privilege of being independent. I can say things that some other people might not be able to say. Our results so far are not being used by anybody in public. What we currently do have is we have a lot of Lightning service providers, we have a lot of custodial wallets that are being used, and we have like a small subnetwork of highly reliable nodes that maintain their notes very heavily. They probe the network constantly, over and over. Like if you run a note, you see a lot of fake payments at your note. And I would argue that many of them are just fake payments where the notes are just trying to understand where is the liquidity currently, or they might rebalance liquidity to where they believe they will need it in the future. And this is the way how they currently improve the reliability of the network. So this is currently happening with trusted providers or with companies who try to see this as a business opportunity, whereas on the protocol and the protocol is highly unreliable still. Right? If I make the comparison to the Internet Protocol, we have the base layer, which is like Ethernet, WiFi, DSL, T, one cable, whatever, these things. And then you have the Internet layer, which is the IP protocol, the Internet protocol. And then on top of this we have TCP, which is like reliability, flow control, congestion control, and then only you have applications like Http, like the worldwide web or email. Right? Whereas on the Lightning network we have the base layer for security that is bitcoin. Then we have the Lightning network that does routing. But the Lightning network protocol itself has nothing that does reliability. So the entire reliability thing is basically solved on the application layer. So we're basically skipping the reliability layer currently. And I am actually more and more strongly advocating for, hey, we should really include this into the protocol. I'm becoming much more vocal these days of saying we really need this. I understand that companies have a business opportunity there, but I think if we don't provide basic reliability on a protocol level on the long term, we're going to be in a very messy situation.
Kevin Rooke Timestamp: 38:59
And when you say basic reliability, do you mean applying this version of multi path payments or multipart payments to all the Lightning implementations?
Rene Pickhardt Timestamp: 39:09
No. So the assumption that I made in my research was that if you choose paths to send to your satoshis that are very likely to succeed, that this will be good for you and you will feel an increase in reliability. Right, but this is something that any LSP can already do. So maybe some people already using my mechanism, but they're just not publicly doing this right. I mean, a lot of LSPs have a lot of proprietary software, so maybe they read my paper and maybe they're just doing this internally. This is nothing that happens on the protocol level. However, there is a huge problem with also the solution that I provide. Because if you send out that many HTLCs, what could happen is one note might just not forward your HTLC for the next 10 hours. What happens then? Right now, if you send out an onion, it's fire and forget. You send out the onion and the only way that you know that it arrives at you. So, for example, let's say we don't have a direct tunnel. So I sent out an onion to you. The only way that I know that it's received is if you release the preemption, right? And if I do a multipart payment on a protocol level, what I do is I split my payment into several chunks. I send out all those onions and you will only release the preempt if all of the onions have arrived. So if only one single note is not forwarding this onion, all of those payments are stalled. Nothing is working for like the next 10 hours. And this is actually something that is happening even with my solution. But I can do nothing about it. This is a protocol issue. Like nobody can do something about this. So, I can give you the optimal way of doing the selection of candidate onions. But even with the optimal selection, we might have bad actors and they might not even be malicious. I mean, they might just be poorly configured or going offline in the wrong moment. There might be several reasonable reasons why a payment might get stuck, right? And on a protocol level, we cannot resolve this right now.
Kevin Rooke Timestamp: 41:12
How could we resolve it in the future?
Rene Pickhardt Timestamp: 41:15
So, we recently integrated tap root into bitcoin. And with Taproot we can basically with Schnor signatures, but I mean, they're part of the temporary upgrade, so that's why I said Taproot. But the Snore signatures are actually the important part here. With Schnor signatures we can do something that is called an adapter signature. And in the Lightning community, this is more commonly known as PTLCs, which are point timelock contracts. This is different to the common htfcs that we are currently using. And with those, what we can do is with tricks that go in the direction of shami's secret sharing, we can basically make payments that we can cancel, right? So we can basically say, hey, look, this onion is not arriving at the destination. Even if it would arrive in an hour, the destination load could not release the premium because it would also need a secret from us, right? So as soon as we migrate the Lightning network to a PTLC world, we have chances to create something that is called cancelable payments, or fixed stak payments, or stak with payments. For that we would also need something that is called onion messages and acknowledgements that onions have arrived. So currently the acknowledgement that the onion has arrived is the release of the pre image. But what we would do then is we would basically, inside the onion, include a return path where the recipient node could basically ping us and say, look, we received your onion. Right? So then I would know, out of my ten onions that I sent, nine are received. The 10th one is not being received in a certain time frame. I will cancel this and I will try another path. Right. In this way, I can make this quicker. Right. So there's a lot of things that have to happen on a protocol level that are currently not in there that I think at a certain amount of time we will need this because otherwise users will feel this.
Kevin Rooke Timestamp: 43:04
Yeah. So what is this path to getting PTLCs adopted looked like? How do you think?
Rene Pickhardt Timestamp: 43:11
Messy. It looks messy. So one problem, and I might have limited knowledge here, is that PTLCs and HTLCs do not seem necessarily compatible with each other. Right. So what this means is currently, when you send out an onion, you negotiate an HTLC on every channel across the path, and those HTLCs all commit to the same pre image. And this makes the entire payment atomic. Right. The intermediate node cannot steal it. But if you wanted to make a path that uses an HTLC and then a PTHC and then an HTLC, this would not work. That's my understanding. Right. Maybe there are some tricks that you can work around that I'm just not aware of, but this is my current understanding. So please, if somebody wants to correct this, I'd be happy to learn. So what we would have to do is, first of all, we would have to make a protocol upgrade where everybody who's running Lightning right now agrees on, this is how we speak with PTLCs. And then, of course, the channels would have to support this. They would have to signal this on gossip, and then I could basically see if I can find a connected path from me to you to only use PTLCs.
Kevin Rooke Timestamp: 44:28
So when you say a protocol upgrade, are you referring to, like, adding another bolt? Is that how that would be expressed?
Rene Pickhardt Timestamp: 44:36
Yeah, so you could see it as a separate bolt. But I guess what is more realistic is that we are adding this to the existing bolts. Right. So the problem with the bolts is even though initially they tried to have separate orders of concerns, there are now a lot of crossindependencies. My feeling is that we probably would just include this to the various bowls. Right. So, for example, the entire channel state machine is explained in both towls. So here you could basically say, hey, there's a message so far that was called update add HTLC. Now we need one that is update add PTLC. But then for the onion routing, we need to understand how we settle this and how we do the on chain transaction and resolvement of channels right. And then we really need to wonder of how can we upgrade this? So yeah, that's going to be a really complicated process with a lot of people working on it.
Kevin Rooke Timestamp: 45:35
I see. Now, when you're thinking about routing payments more generally on the network, and you have a few different constraints you can optimize for, one could be reliability, it could be latency, it could be fees. What do you think we should be optimizing for in the Lightning Network? I know every different party will have a different answer here, but what do you think the Lightning network needs more of today?
Rene Pickhardt Timestamp: 46:05
Yeah, so, I mean, traditionally, everybody was optimizing for fees and the reliability was really bad, right? I mean, unless you did really like 100 satoshi payments, nothing worked initially. Right. I mean, it became better with people maintaining their notes. But even rebalancing is kind of like tricky because if everybody is, like, routing across low fee routes, then your rebalancing circle will obviously be more expensive because otherwise the node would already have taken that path. Right. So you would actually pay for rebalancing, which was like a long time a discussion. So optimizing solely for fees, I believe, is something that is not sustainable in the long term. However, if you only optimize for the likelihood to succeed or reliability in that sense, this is also tricky because then somebody can charge like, really high fees if you just got to pay them. Right. So this is also a great so I think by the end of the day, we need a combination of these two features, potentially also latency, which you could argue is a subpart of reliability. But these are currently the three main features that I see. So the reduction of uncertainty about the liquidity, the latency, and the fees that you're going to pay. Right. And you need to find some smart way of combining this into an optimization problem.
Kevin Rooke Timestamp: 47:20
I hope you're enjoying the show so far. I just want to give a quick shout out to our sponsor, Voltage. Voltage is the industry standard for Lightning Network infrastructure, creating layer two applications and services on top of Bitcoin starts with Voltage, where you can spin up nodes, get access to liquidity, optimize your node, and much more. Voltage is leading the way as the next generation provider of Lightning network infrastructure. And if you want to get a free trial and start using Voltage today, you can do so at Voltage Cloud. So when you look at the Lightning network today and you look at those three factors, reliability and fees and latency, how does Lightning network compare today to traditional fiat payment rails or existing rails?
Rene Pickhardt Timestamp: 48:07
Yeah, so I mean, even today, it depends on what you optimize for already. So I have recently released the software that goes along with the paper that we published last year, which you can find on my GitHub repository under the name Pickup Payments, because a lot of people started to calling it this way and last month I finally decided that I would also like go along and call it this way. And what you can do with those is you can basically already optimize for at least fees and reliability. I'm working on also including latency. And when you do the simulations, what you see is if you optimize solely for reliability, you're going to pay a fee of roughly 4000 ppm which is like zero 4% with the fees that are currently on the network. However, if you optimize mainly for fees, you pay something like 1500 ppm, right? So that's already like mainly only a third of the price. But you have a lot more failed attempts, a lot of more retry, right? And then you do something in between where you at least according to my simulations as of right now, pay something like 2000 ppm, which is zero 2%. The reliability is obviously not as good as using something like Visa. Visa basically always works as long as your internet connection works. Of the point of sale system perspective. Here we have much more retries and also the latency is higher. So currently for a single onion or for a single payment to settle the median time, even when using probabilistic path finding, I think Christian measured something like 5 seconds, five point something seconds and that's the median, right? So 50% of the payments take more. I mean, this is something where I used to say there's room for improvement. This is also why I want to look at latency. And for example, we can improve latency, I think, quite a bit if we have L two terms in the future because the round trip time to negotiate a new state will be reduced, right? A lot of these questions are also related to what we implement in the software, right? So for example, right now all the Lightning implementations optimized for fees. So the nodes are actually trying to underbid each other with fees. But as soon as our solution is being utilized more, what will happen is when you have a large channel that's highly reliable, you can actually, from the game theory already charge a higher fee, right? So then you certainly start to price for liquidity and for a reduction in uncertainty, right? So I would guess that the fee market will have a very different dynamic over the next year when the people who are sending payments actually change their dynamics, right? The same way, including latency into the feature. Because now you might like currently my solution even would prefer a very slow channel that goes from Norway to Australia just because it's a large channel and it's a cheap channel, right? But the latency is obviously really bad, right? And I mean, everybody naturally understands that I should probably take the channel from Norway to Germany that is almost the same size and a little bit more expensive, but is much faster to try right. A lot of this dynamic will actually change if we optimize for different things, because then the people will charge their fees differently, right? So it's very hard to foresee these things in the future.
Kevin Rooke Timestamp: 51:39
Do you think it's realistic to expect higher fees in the future on Lightning?
Rene Pickhardt Timestamp: 51:43
Yes or no? Yes and no. So I know, for example, that the zero fee routing guy was in your show. I haven't watched that episode, but I have talked to him quite a bit and I find it very intriguing what he's doing, right? Because he's basically saying, hey, look, all the implementations are basically optimizing for fees right now. So I charge zero fee, everybody's routing through me. All the people who are rebalancing are using my note. I mean, this is great. And how does he earn money? He basically asks for the Ppm on the inbound liquidity. If you think about this, and he probably said this to you, this is very similar to how current payment systems work, right? Because if you are a shop and you want to use this, you buy your inbound liquidity at like 2000 Ppm, 3000 Ppm, whatever is the price, right? I mean, it's very similar to installing a point of sale system where you as the merchant, pay a certain fee in order to be able to receive money, right, and the customer doesn't see the fee in the moment of making the payment anymore. Right. So what I'm saying is, what could very well happen is that the fee on the Lightning network, because of behavior like this, which to some degree is very rational from the service provider, because if I open the channel with you and I charge a Ppm for the routing, the routing actually have to take place, right? That's a risk on my end. But if I charge zero Ppm and you pay me when I open the channel, the risk now is on you. You as a merchant, you're going to be like, yeah, I'm going to sell products. That's your economic risk. And for me, it's really great, right? So what I'm saying is the game theory, I think, has not played out yet of what we do. So it might very well end up in a world where we just don't pay fees for routing at all. But of course, indirectly the fees are being paid at general opening and from the merchants. And then of course, the consumers pay this in higher price than the stores, right? I don't know.
Kevin Rooke Timestamp: 53:36
That's an interesting possibility where, yeah, it could be just a merchants footing the bill, almost, or just like a different dynamic between the people who are paying and people routing, maybe doing it for free.
Rene Pickhardt Timestamp: 53:49
The only thing I want to say here is right, I'm not saying that this is certainly how the future will pay out, because there are certain reasons why I also see that this is problematic, but I just like trying to describe, but it's really hard to see this. The only thing that I am really certain about is that it will not stay in the way how it's currently being done. I think that's very inefficient and I think the market will mature.
Kevin Rooke Timestamp: 54:13
Okay, so now on the topic of zero fee routing, let's get into zero base fee.
Rene Pickhardt Timestamp: 54:20
Kevin Rooke Timestamp: 54:20
I'd love to understand what that concept is and exactly how it works.
Rene Pickhardt Timestamp: 54:25
Okay, so as I said before, I was basically the first person, then later, together with Jeff, of course my coauthor to extensively study and provide the solution. I mean, maybe people have studied it before but didn't publish the solution of this optimization problem of routing satoshi from one node to another, which is this famous Mincostro problem that is basically used everywhere in logistics and finance, everywhere. And it turns out that the solvers for minimum cost for problems, their runtime depends heavily on the cost structure, on the mathematical structure of the cost function. So if your cost function is a linear function, then you have very fast solvers. But if the function is non linear and nonconvex in particular, then the problem is known to be NP Heart. There is a huge bounty to solve NP heart problems in an efficient way. I think currently no same person and research assumes that this will ever be possible. It is an open research question if it is possible, but I think the common assumption is that it is not. Right? So if you have a base fee, the optimization problem for sending notes to send a payment and to solve this minimum cost for a problem will become NP heart. Right? Which means it's very hard to solve. And just to give you a firm grasp, when we initially understood the main cost for problem and we wrote down our first algorithm, we had an algorithm that was able to run on the convex cost function and took us 6 seconds to just solve the problem. Right. So if you wanted to make a payment, you would have to wait 6 seconds until I could tell you these are the candidate paths that you try and then you still send them out and you still have a certain chance that they fail and then you have to iterate on this. Right? So it's a very poor experience. Some people who were very critical about our results were like this takes too much time, it's a nice result, but it's unusable. Now, with a lot of tricks I have been able to get this runtime down into like 100 milliseconds, which is very comparable to what nodes currently do to generate candidate paths. Right? But mine are like much closer to the optimal solution. But the way how I do this is I only compute this on the subnetwork that charges zero base fee. Like if I would include the channels that have a base fee, I would be completely skewed. Like I couldn't do it. Right? So what we are basically saying is, look, there is a market where you offer routing as a routing node, and of course you want to offer customers to choose the best offer on the market, but they can only do this if there is no base fee. Right? And because reliability is yeah, sorry.
Kevin Rooke Timestamp: 57:18
If there is a base fee, it gets in the way of determining what the correct path might be or the outphone path might be.
Rene Pickhardt Timestamp: 57:23
Yeah. So the algorithm doesn't work. Like, the algorithm that we provide will just not produce a result. And if we were to provide an algorithm that can handle the base fee, the runtime would probably go into the hours. Wow. It's like you would get a ridiculously low runtime for principal mathematical reasons. All the proofs have been conducted already 30 years ago. Right. We only discovered this by understanding the problem and then digging out the relevant research that was already there. So this is like, a well known mathematical problem. This has not been rejected for many years. So we kind of believe this is true. And I think Stefan actually went through the proof. Like, he's very theoretical on those. I was like, Yeah, I think this is reasonable. So what we're basically saying is, if we want to have this routing game to be like a game that works, maybe not operators and routing not operators should offer the option for zero base fee so that people can do this, what I call a selfish routing, which means finding the best candidate path according to their optimization goal.
Kevin Rooke Timestamp: 58:27
Now, it seems like this is a better way of discovering paths on the network. So my next question is, why is this not already implemented? Why is every node not using zero base fee today?
Rene Pickhardt Timestamp: 58:43
Honestly, that's a question you have to ask to the people who refuse to update their base. For example, if you go to Cast and Auto, I don't know if you have talked to him in the past, but custom Auto, I think he's running one of the most famous, like, flat notes, so to speak. Right. So Castin Otto, he's just a computer scientist who likes Lightning Network and who's just rebalancing heavily. And I think he likes to play this game of, like, having a high score and all those Lightning Network node explorers. And on his website, he basically has some instructions of his experience of, like, how do you run a Lightning node? And he's like, this is the channel size you should do. This is when you should rebalance. These are the PPMS that he charges. This is how you like dust stuff. And then he comes to, like, base fee, and he's like, base fee set to zero. There's research that explains it very well. It's obvious, right? It's a no brainer. Right? And a lot of people have followed not only question, but also followed our research and followed our results. But obviously if you have an option that is like, hey, do you want to change something? And it almost seems like, and I don't want to get too political here, but like gun laws or abortion rights, those are these questions that are very easy to polarize. Right, because you're like, yes, I like this, no, I don't like this. Right. And you can have a very firm opinion on those without going into all the difficulties that are behind this. Right. And obviously the people who set zero base fee are just trusting in many cases us and our results or the research results that have been there in the past. Right, so, yeah, I guess it's just like a process of how society finds a certain consensus on something or agrees on something. That's my feeling. I'm not sure.
Kevin Rooke Timestamp: 01:00:26
What was the idea of having a base fee as this kind of default setting before your research came out?
Rene Pickhardt Timestamp: 01:00:33
Yeah, so that is actually something that we asked us ourselves. Right. So I couldn't find anything, so I went on Stack Exchange, which is a place where you can ask technical questions and I was asking why is there a base fee or why was the base fee included to the protocol? Is there a good reason for it? And the question wasn't answered for quite some time. And then I pinged it, like I said it basically to a few Lightning developers and be like, hey, do you have a thought on this? And then Rusty Russell from Blockstream was kind enough to basically answer that question and say we had a discussion at that time. Some people wanted like no fees, some people wanted fee. Something like blowing up the answer. I think he was much shorter on Stack Exchange, but he basically said there was a discussion around this and regular payment providers usually have a pays fee and a fee rate, so it seemed reasonable. So we did it. It's basically like an arbitrary choice that we did. Right.
Kevin Rooke Timestamp: 01:01:25
Is that to reflect when you say regular payment providers, you mean like the Visa may charge plus $30?
Rene Pickhardt Timestamp: 01:01:34
Yeah, something like this. Exactly.
Kevin Rooke Timestamp: 01:01:36
Interesting. And so now you can do it without the base fee at all.
Rene Pickhardt Timestamp: 01:01:40
Yeah. On the Lightning network, stuff gets a lot more easy if we have linear problems and if we remove the base feed, it's a linear problem. And since we have so many problems on the Lightning network anyway, I highly advocate for let's remove it. And to be frank, I mean, by now so many nodes have set their base fee to zero that for example, in the software that I provided, I only compute all the candidate paths on the zero base fee graph. If you want to earn a routing fee, you better set your fee to zero because I'm going to sling large payments with my software over the network. And on large payments you earn a really nice yield on your Ppm right? Like if you have a Ppm of, let's say, 500, and I sent like 10 million satoshis along this, you earn a nice juicy 5000 satoshi if I calculated this correctly. You know what I mean? Right, yeah.
Kevin Rooke Timestamp: 01:02:36
So now if the entire Lightning network, I think today, 40% to 47% of the Lightning network nodes and capacity are using zero base fee. If everyone was using a zero base fee today and we could just flick a switch, do you think there would be any changes to reliability on the network from a consumer end? If I'm just using the network as a regular consumer, would I notice any changes?
Rene Pickhardt Timestamp: 01:03:04
So what I already tried to explain last year is initially this change for zero base fee doesn't change anything at all. Right. I remember on the very first day when the paper came out, some people were like, yeah, I set my base feet to zero and I have the feeling I see more routing traffic. I'm like, this is not how it works. Because obviously as a sender, you need to have the software that actually computes them in cost flow. And it's only until this week that I actually, for the first time, release software Incomplete, where you can play around with it. And even for that software, it's very hard to do it if you're not an expert. Like I deliberately made the software a little bit more complicated to use so that only experts initially would start doing this. So I highly encourage all of them to do it. Right, so this is a process, right, but as soon as the wallets and the router, like the sending nodes use this kind of software, then of course having more channels with zero base fee is more liquidity on the network and changes something, right? So the zero base fee thing from day one on was always like a long term commitment to support routing. Initially somebody said, renee, your results are nice, but we need zero base fee. But unfortunately, 95% of the network don't have zero base fee, so your results are pointless. Well, people can change that and people did. But to answer your other question, is it better if we have more liquidity on the network? That's a really tricky one. So if you look into game theory, I think already in the 70s there was this German mathematician called Braess, and he came up with this thing called the Braess Paradox. I don't know if you have ever heard about this, but maybe your listeners haven't. So let me elaborate on this a little bit. So the Braess Paradox is this phenomenon that when you have a street network and you have traffic in the street network, right? Like you might, for example, have like a big city and people commuting to the city every day, you can measure the demand of traffic. You could observe that certain roads are stuck every morning and then somebody could have the idea of hey, let's build a bigger road. Okay, so this obviously has happened in the past. What has also happened in the past is that people have built a bigger road and afterwards we observed more congestion which is completely counterintuitive to in first order. Right? Because you would say, hey, if you have a bigger road, you have a highway, you can drive faster. Well what you neglect there is the selfish behavior of the people who say oh this is a bigger road, I will take maybe a slight detail to take the faster route. Right? So now more traffic actually is being attracted to this road and you have congestion because it's overloaded again. Right. So we had situations in the past where big roads actually have on purpose a lower speed limit or maybe I even blocked on purpose to make flow control interesting. So this is very similar on the Lightning network, right. If you don't think about the problem hard enough you might say I just open a large channel and that's very good for the network but the large channel might be very attractive and might get jumped and then of course some people come and say yeah, but then I can increase the fees. Right? Sure it's a game but what I'm saying is more liquidity is not necessarily a good thing. It may be right, but it depends. It's really complicated and to me actually it's an open research question because this entire Braess Paradox phenomenon doesn't translate specifically or directly to how Lightning works. And luckily I have two summer of Bitcoin projects or two students for one summer of Bitcoin project working on this for the next twelve weeks. So I hope you really get some results there.
Kevin Rooke Timestamp: 01:06:48
Interesting. So in this example of the roads, does expanding the road make it actually a worse experience from a speed perspective or is it just like not as much of an improvement as you would expect?
Rene Pickhardt Timestamp: 01:07:03
No, it can make it worse and it has been observed in several cities, in several countries, it has been observed over and over again and I think this like German mathematician Brussels was the first to formally prove why this can happen. Like under which conditions does that happen? So you can actually even predict this to happen and it's like as I said, it's completely counterintuitive. Right. You have congestion, you just make a second lane, you allow faster traffic and now you have even more congestion. Like how did that happen?
Kevin Rooke Timestamp: 01:07:30
Yeah. So now if you were the traffic controller of the Lightning network and you had full rain, full authority here, how would you try and lay out the grid or this kind of network to optimize for payment flows?
Rene Pickhardt Timestamp: 01:07:48
So you are asking one of the best questions to some degree that I have been asked on a podcast ever. I have to say this because I am not in this position and I should not be in this position. The entire idea and purpose of the Lightning network is that we want to build a decentralized network, right? And the interesting observation is when you have this network and let's assume you have this godlike person who could do all this flow control, they could basically study this problem and do flow control and they would basically tell people, hey, this is how the network looks like, but you don't take the highway, like, even though you wanted to, but today you're going to take a detour so that the highway for the others does not congest. Right? So what such a king of the network would do, it would deliberately give some people a solution that is not optimal for those people to increase the overall common welfare or the social welfare. This is how it's being called in the literature. Whereas on the Lightning network, we have the situation where everybody just does this selfishly. And if everybody does the selfishly, the solution that is being found is usually not the global optimum that I could find if I could control everybody. Right? And the difference is what we call the price of anarchy. Like, it's a term in scientific literature and you can actually, for many networks, quantify what is this price of anarchy? Right? And what I currently don't know is how high is the price of anarchy in the Lightning network? I just don't know this that's part of the Summer of Bitcoin project to find out. Because if the price of anarchy is small enough, then we are very happy, like, we're very happy to pay a certain price of anarchy if we don't need this, like, central controlling kind of person to find good routes. But if the game of the Lightning network of routing is structured mathematically in a way that the price of anarchy is arbitrarily large, then of course it's really, really bad because the network is going to naturally congest.
Kevin Rooke Timestamp: 01:09:55
Interesting. Are there parallels here between what you see in maybe, like, capitalist societies like, I think the US. Versus China, where it's like in the US. Everyone's kind of like acting in their own self interest is very, like, liberty first freedom. And everyone has their own strong property rights and we have the ability to look after ourselves and all of a sudden, maybe it's not an optimal situation if there was a king ruling over the land, but everyone has the freedom to do what's in their self interest versus another society where it is ruled by a dictator and there's strict rules on what you can and can't do and someone's like overseeing and planning the network. Is that a comparison that I can make?
Rene Pickhardt Timestamp: 01:10:45
Do you think so? I have lived in China for almost two years, and I have lived in the United States for one year.
Kevin Rooke Timestamp: 01:10:52
Rene Pickhardt Timestamp: 01:10:52
So I kind of know both countries, and I can tell you there are severe differences. Severe differences, right. For example, in china. When I lived there initially, there was no KYC. When you entered an Internet cafe, like, you could just go there and sit down on a computer and surf around. But while I was living there, they basically released a law that now there is KYC. Right. And when I wanted to go to the Internet cafe, if they didn't see my passport, I would not touch a computer. This law, I was traveling in this moment when they released this law, it was basically every day in an intel cafe. And I was traveling not in Beijing, I was traveling very far in rural areas. This law was active within a couple of days, like, everywhere to some degree. Really amazing and scary how quickly the Chinese people can enforce a law. Right. And there's hardly any protest. It's just happening. It has been decided and then we're going to go and comply. It's probably been decided because it's good for everybody. That's the assumption. Everybody just goes along in the United States and I think also in Germany. I mean, if you, for example, think about the Corona situation, like, vast amounts of protest, and of course you can say this protest is good because you have your rights to object and you have your liberty. Right. But I think I'm not saying something wrong if I say that this protest comes at a certain price. I'm not saying that this price is like, too high or too low. Right. I'm not even in the position of, like, evaluating this because those are generally hard questions. I mean, of course you find people who are very much proliberty who would say this price cannot be high enough. Right. And you would find people who are very, like, socialists who say it's, of course, way too high. But I think people could observe that these protests have a certain price connected to them. Right? Yeah.
Kevin Rooke Timestamp: 01:12:51
There's trade offs.
Rene Pickhardt Timestamp: 01:12:52
There's trade offs. Right. And I mean, obviously, if you look into economic theory, there are things that are called market failure. Right. Similar, you have state failure. Right. And you could also argue that in communist countries you might have state failures. Right. These are generally hard questions, and I don't think I'm in the position that I could answer them in one way or the other, determinedly. It's, like, literally hard. I can just say I have seen both sites and I have seen pros and cons on both sides.
Kevin Rooke Timestamp: 01:13:23
Yeah. So now if we look at the Lightning Network as this like if we use America as an example for the Lightning Network, where it is distributed decentralized, everyone can choose to take their own self interest into account. Do you see someone building a solution that is like a digital payments network first, that is structured in a way where there is a ruler, a single ruler? I think Visa and some of the other networks that have existed in the past maybe are not entirely digital networks in that capacity. But do you see someone else stepping in? Maybe this is a CBDC, maybe this is another blockchain. Do you see there being a in this example of using the US. And China? If the US. Is the Lightning Network, will there be a China?
Rene Pickhardt Timestamp: 01:14:22
I think what I have learned while living in China is and I was criticized very heavily by my German friends when I said this in public, right, but I will say this in public again because that was my experience democracy is nice, but what is far more important than democracy is actually that you have checks and balances, like a division of power, right? You have certain institutions that control each other and it doesn't necessarily that was my statement that I was criticized being heavily for that it doesn't necessarily have to be in a democratic session as soon as you have these checks and balances and a working system of leveling the gaming playing field. Right. So I think this is something that attracts me very much to the Lightning Network and Bitcoin because this is also a very neutral technology. And what my feeling is a lot of people have difficulties to understand. I'm not talking about Bitcoin as I'm mainly talking about no coiners here is they intuitively think that a neutral thing might be something good because it's neutral, it's nice. But obviously neutral can also allow bad actors to be very powerful, right. If you look, for example, at the Internet, it's also decentralized, or at least the World Wide Web, it's also a permissionless decentralized system. I mean, everybody can run a Web server and everybody can run a Web client, right? Yet for people it turned out to be very useful and helpful to have things like Google, Facebook, Twitter, all those heavily centralized services where people later then go and say, hey, cancel culture. You censor something on Twitter like, yeah, but why did you use it in the first place? Like it was always clear that this could happen, right. So what I'm trying to say is if you have a permissionless system, it's in my opinion almost inevitable to that some centralizing powerful powers will come, forces will come right. Now in Bitcoin, you could say those are the exchanges to some degree the gatekeepers and enlightening. I would assume at some point in time this will be service providers.
Kevin Rooke Timestamp: 01:16:39
Yes. Now, I want to finish this off. I know we're running out of time, but I want to finish off with some limitations of the Lightning Network. And I want to understand like we talk about how great Lightning Network is on this show all the time and every guest comes on and there's some fascinating projects being built, but I know the network is not perfect and I know you're doing some really technical research and figuring out some of those deficiencies and flaws and things that can be improved and would love to just finish off with just a high level of like what are some of the things sticking out to you that the Lightning network needs to fix, needs to improve, needs to build upon?
Rene Pickhardt Timestamp: 01:17:15
Yeah, so I think most of them I have already mentioned during the show. Right? So for example, we don't have reliability built into the protocol. Currently service providers are trying to fix this for us. But this of course during the entire discussion that we just had about centralizing forces gives them a tremendous power because they solve a problem that the protocol would not solve. Right, so I would argue on the protocol side we need more reliability for this to be useful for people. The other thing is this entire game theory of like selfish routing and Ross paradox, right? So you could have like natural congestion emerge, like maybe if the Lightning network reaches a certain size. I mean it's certainly better than not having a Lightning network and doing everything on chain. But I am not sure how far this will actually scale and how the behavior of people has to be and how the routing nodes are like helping users to do the flow control. Right, so I think there is a certain limitation. If you go to my Git repository where I published the code for making like really fast payments with large sums, like large amounts, what you will see is one way of achieving this is before I do the computation, I do not only throw away the edges that don't charge a base fee, but I also throw away edges that are either too small or too expensive with a fee rate because they already know they will most likely not be part of the solution. But one consequence of this is that the larger your channels are, the more liquidity you provide, the higher Ppm you can actually charge to still be included into the computation, which gives us the rich get richer effect and centralization effect because you just have this economic of scale. So I would argue there are a vast amount of limitations that may be built into the protocol that we might not have seen yet, the stack of statements that we should work on and that we really need to do more research and try to fix.
Kevin Rooke Timestamp: 01:19:15
Right. This has been a fascinating discussion, I learned so much and I'm glad we got a chance to do this. Where can people go to learn more about you and the work you do?
Rene Pickhardt Timestamp: 01:19:28
So I have my website, which is Ln rene Picard De. I usually list at least all the papers that I write and the results that I do. Obviously you can go to my GitHub repository where I'm trying to once in a while publish some code. Right? So as a researcher I'm often looking at theoretical things and then I might write a paper eventually or I might provide some code. But I'm not like hacking on a daily basis. So that's a little bit of a pity. Usually I share a lot of my thought process on Twitter, so it's actually really funny. So, for example, with this optimally reliable payment flow paper I was sharing on Twitter that I found the optimal solution, and I was basically asking, hey, should I sell this to a company or should I found a company? And people were throwing VC money at me and were like, hey, do you want to do this for us? Because everybody understood it's a pain point, right? But the funny thing is, what nobody knew is that I already within the last month, had basically sent six tweets where I basically hinted to all the research that was necessary to do this. So in our paper, we actually in the appendix listed all those tweets and basically kind of like, hey, look, the information was out there all the time, listened carefully. I'm not saying that everything that I wrote about on Twitter is reasonable, right? Sometimes I'm just also misleading. But at least my thought process is kind of like transparent there. Yeah.
Kevin Rooke Timestamp: 01:20:54
Awesome. Well, thank you for taking the time and hope you enjoyed.
Rene Pickhardt Timestamp: 01:20:58
Thank you. Sure.