Building a LAPP Series: lnsolve.com
You must be aware that an open platform like bitcoin comes with its ability to create applications. In this post I'll explain the how I built lnsolve.com using lnpay.co as my bitcoin node.
First we need to understand what we're building. We're building a website where users publish problems and assign them bounties. Other people can solve those problems and be rewarded that bounty. These bounties are bitcoin transferred via the Lightning Network.
This involves handling bitcoin programmatically. Being able to deposit and withdraw on demand while also function with the website design.
With bitcoin's Lightning Network It's much easier and works on global scale. Anyone with an internet connection and a basic Lightning wallet like Phoenix wallet can participate. In lnsolve the only "registration" is a username password. Which in itself isn't necessary.
To do this, First signup at https://lnpay.co/home/signup and then go to https://lnpay.co/wallet/dashboard and create a wallet.
Here you need to decide something - Do you want each of your website's customer to have a separate wallet or want to put all funds into one wallet? In lnsolve.com I created one master wallet and stored everyone's funds in that.
wallet created for demonstration
At the time of writing, lnpay.co supports mainnet only.
Do not share these Wallet Access Keys. You'll be using them to communicate with your wallet.
lnpay.co provides an easy to use API to communicate with your wallets. You'll be using that API from your server to create invoices and so on.
Your website will be hosted on a web server. That can be on premise or on cloud. I cannot teach everything about web development from scratch in one post. So I'll be assuming you have some experience.
I have used this tech stack:
- Infrastructure: Amazon AWS EC2 machine
- Server Application: Express.js (with NGINX reverse proxy)
- Database: MongoDB
- HTML rendering: Pugjs
- Bitcoin Node: lnpay.co
You can setup your node project with
npm initTo know more about setting up mongodb you can refer this: https://linuxhint.com/install_mongodb_ubuntu_20_04/
Our express server initialization looks like this:
server_domain = "https://localhost:10001"
// server_domain = "https://lnsolve.com" // use in production
// setting up SSL
var fs = require('fs');
var https = require('https');
var privateKey = fs.readFileSync('./localsslkeys/cert.key', 'utf8');
var certificate = fs.readFileSync('./localsslkeys/cert.crt', 'utf8');
var credentials = {key: privateKey, cert: certificate};
// importing express
var express = require('express')
var app = express()
// setting up cookie and session
var bodyParser = require('body-parser');
var multer = require('multer');
const path = require("path");
var upload = multer();
var session = require('express-session');
var cookieParser = require('cookie-parser');
// setting up render engine
app.use(express.static(__dirname + "/views"))
app.set("view engine", "pug");
app.set("views", path.join(__dirname, "views"));
// setting up cookie and session
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(upload.array());
app.use(cookieParser());
app.use(session({secret: "secret"}));
// behold - our server
var httpsServer = https.createServer(credentials, app);
// setting up MongoDB
const MongoClient = require('mongodb').MongoClient;
const ObjectId = require('mongodb').ObjectId;
const url = 'mongodb://localhost:27017';
const dbName = 'solveforme';
// Create a new MongoClient
const client = new MongoClient(url);
Woah! That looks very scary, but do not worry its really just some copy pasting and filling up details like secret seed.
Once we're done initializing, we connect to our database and start listening to end points on our server:
// use mongodb object to connect to the database
client.connect(function(err) {
// our database object - We will use it to query our database
db = client.db(dbName)
// Now we can create our express endpoints
app.get('/', (req, res) => {
res.send("you've made it!!")
})
})
Focus on the app.get('/'.. line - Every time someone interacts with your server they send a https request to the url paths defined in here. '/' just means root path. So if your website's domain is example.com, it'll send "you've made it!!" when you visit https://example.com/.
As our focus is on Bitcoin and Lightning network, I'll primarily be discussing endpoints specific to bitcoin. If you wish to see more, the entire project is open-sourced at https://github.com/ParmuSingh/lnsolve.com
When ever you submit a problem to lnsolve.com you also pay the bounty that is attached to the problem you submitted and is awarded to the person who solves it.
Let's look into how that works.
Q) Why create invoices?
A) Invoices are used to pay you. User will use this to pay you which then triggers "something". In our case, "something" means submit the problem with the payment amount as it's bounty.
Go to lnsolve.com/submit_a_problem and test posting a problem. You will see that you're prompted with a QR Code. What happened is when you clicked on the "Post Puzzle" the website submitted the problem data (like it's title, description, bounty value) and the server returned with a lightning invoice.
To explain better lets look at it step-by-step:
- User fills problem details and clicks on "Post Puzzle"
- Website takes the problem details and submits it to app.post('/submit_a_problem') endpoint of our server
- Server verifies that the data is correct and pushes it into database as "Unpaid puzzles". (It'll be moved to "Open Puzzles" once the payment is done).
- Server then requests lnpay.co for a lightning invoice for the bounty amount and returns it to the user along with the problem's id.
- When user receives the success message and the lightning invoice. It creates a pop up and shows that request in the form of a QR code.
- Client periodically checks with lnpay if the invoice is paid. If yes redirect user to their problem's web-page.
- When invoice is paid lnpay.co tells our server via webhook that that specific transaction is complete and the server publishes the problem.
Now let's look at the code for each step.
When user clicks the "Post Puzzle" button
post_problem_with_bounty() is calledroot_domain = "https://"+window.location.href.split('/')[2]
function post_problem_with_bounty(){
title = document.getElementById('title_field').value
description = document.getElementById('description_field').value
bounty_field = document.getElementById('bounty_field').value
// check if fields are not empty
// The actual request client sends to server
$.post(root_domain+'/submit_a_problem', // url
{ title: title, description: description, bounty: bounty_field }, // data to be submit
function(data, status, jqXHR) { // success callback
console.log("response from server:")
console.log(data)
if(data['message'] == "success"){
// >>>>> invoice received <<<<<.
document.getElementById('post_problem_btn_message').style.display = 'flex'
newProblem_id = data['problem_id']
// show qr code
show_ln_request(data['lntx']['payment_request'])
}else if(data=="less_bounty"){
document.getElementById('post_problem_btn_message').style.display = 'none'
document.getElementById('message').style.display = 'flex'
document.getElementById('message').innerHTML = 'Bounty needs to be atleast 1000 sats'
}else{
document.getElementById('post_problem_btn_message').style.display = 'none'
document.getElementById('message').style.display = 'flex'
document.getElementById('message').innerHTML = 'Some problem occured'
}
}) // end of post request
}
$.post()sends a post request to /submit_a_problem endpoint and gets the lightning invoice and shows it as a QR code using show_ln_request(). If you wish to see how the pop up logic works you can check it on https://github.com/ParmuSingh/lnsolve.com/blob/master/views/submit_a_problem.js#L65Note that the above code runs on client's browser and not on your server.
But what does our server do when it receives the request? It submits the problem as unpaid bounty and returns the invoice:
app.post('/submit_a_problem', function(req, res){
if(!req.session.userAuthenticated){
res.send('not authenticated')
return
}
if(!req.body.title || !req.body.description || !req.body.bounty){
res.status(400);
res.send("Invalid details!");
return
}
bounty = Number(req.body.bounty)
if(bounty < 1000){ // minimum bounty
res.send({message:"less_bounty"})
return
}
var timestamp = new Date()
// this problem goes into unpaid problems
var newProblem = {title: req.body.title, poster: req.session.userName,
description: req.body.description, timestamp: timestamp,
answers: [], isSolved: false, isBountyPaid: false,
bounty: bounty};
// Here we multiply the bounty with 1.01 because lnsolve takes 1% fee to support server cost
bounty = Math.ceil(bounty * 1.01)
// get invoice
helpers.request_invoice(res, bounty, db, newProblem, memo="posting problem in lnsolve")
}) // app.post('/submit_a_problem')
In the above code,
helpers.request_invoice() gets the invoice from LNPAY and sends it to user// THIS IS WERE WE GET THE INVOICE
module.exports = {
request_invoice: function (lnsolve_res, amt, db, newProblem, memo="lnsolve"){
var headers = {
'Content-Type': 'application/json'
};
var dataString = '{"num_satoshis":'+amt+', "memo":"'+memo+'"}';
var options = {
url: 'https://lnpay.co/v1/user/wallet/waki_aP85gjJ2AznCmldc0EZpHjA/invoice',
method: 'POST',
headers: headers,
body: dataString,
auth: {
'user': 'sak...', // your full API key here
'pass': ''
}
};
// This function executes when server recieves invoice from lnpay
function callback(error, response, body) {
// body is a JSON that contains data about the invoice. Like payment request, txid.
body = JSON.parse(body)
txid=body['id']
// adding txid to problem. It'll be useful later when we're publishing the problem using the txid.
newProblem['ln_txid'] = txid
// pusing problem to unpaid_puzzles collection in mongodb
collection_puzzles = db.collection('unpaid_puzzles')
collection_puzzles.insertOne(newProblem, function(err, docInserted){
// Now if webhook recieves wallet_receive event with same txid then we publish the puzzle
console.log(new Date() + ": new problem received")
lnsolve_res.json({problem_id: docInserted['ops'][0]['_id'], lntx: body, message: 'success'})
})
}
request(options, callback);
}
}
Here
waki_aP85gjJ2AznCmldc0EZpHjA is wallet specific. So go to https://lnpay.co/wallet/dashboard to see your wallet's "Wallet Invoice" code. Another thing is sak_... which is your API key which is specific to your LNPAY account. Go to "Developers" tab to get it.LNPAY offers many wrappers for Javascript, Node.js, Python and Go. I have used the API directly instead of using wrappers. You can see these wrappers at https://docs.lnpay.co/#client-sdks
From here, to keep this post small, I'll only show snippets for bitcoin related code only.
Q) Why check transaction status?
A) When the client is shown the QR code, the browser periodically checks with LNPAY if that payment is successful. If it is, the browser can tell user that the payment is complete and redirect user to problem's page.
You may have wondered why we're not sending requests to LNPAY directly from client. The reason we've kept all bitcoin related code server side is because the client can modify the javascript code and change values, like request lower bounty than what's being submitted. This will also be a security concern when withdrawing because the user can possibly modify the code to withdraw more than what the user owns. If you had decided to make a separate wallet for each user on LNPAY this'll be less of a concern.
Though when the client is polling to see if the transaction has completed - that is something we can safely keep between client and LNPAY only.
let publicApiKey = 'pak_...'; // Your *Public* API key
LNPay.Initialize(publicApiKey);
// takes txid as input
function check_tx_status(txid){
let lntx = new LNPayLnTx(txid);
// calling getInfo function
lntx.getInfo(function(result) {
console.log("tx settled: "+result.settled)
if(result.settled == 1){ // result.settle is equal to 1 when transaction is complete
// if the invoice is paid redirect user to problem id
window.location.replace(root_domain+'/problem/'+newProblem_id)
// and tell user the payment is complete
document.getElementById('payment_message').innerHTML = "Payment settled. Redirecting.."
}
}
);
}
You can get "pak_..." from LNPAY's developers tab. Make sure to not use the private API key here.
To make this function call periodically use
setInterval(check_tx_status, 2000, txid). This calls check_tx_status(txid) every 2 seconds.<script src="https://unpkg.com/[email protected]^0.1/dist/lnpay.min.js"></script>Q) Why use webhooks to know if transaction is complete?
A) The server uses this information (that comes from LNPAY) to mark the submitted problem as paid and publish it as available problem. We use webhooks instead of API because they're more efficient than polling.
Once the user completes the payment. LNPAY becomes aware of it, and then alerts your application server about it. Your server has to be made aware of it so it can publish your problem and make it available for everyone to solve.
To setup your webhook go to https://lnpay.co/developers/webhook > Create Webhook. Here your Endpoint Url is the url where LNPAY will send the alert. You can select Wallet Receive and Wallet Send so your server is alerted whenever money is deposited or withdrawn from your wallet.
This "alert" is a https POST request. If you don't know much about webhooks I recommend reading about it: https://snipcart.com/blog/what-are-webhooks-explained-example
If you're building locally and haven't hosted anywhere to listen to webhooks I recommend using webhook.site to test them out.
screenshot of example webhook reqeust from lnpay
You can decipher the requests and figure out where important fields are. I have done that so you could use my code:
app.post('/webhook', function(req, res){
// req is the request sent from lnpay webhook
// You can get these fields using these paths
var created_at = req.body.created_at
var id = req.body.id
var event = req.body.event
var data = req.body.data.wtx // data
helpers.log("event: "+JSON.stringify(event))
helpers.log("\ndata: "+JSON.stringify(data))
// is the alert about deposit? (when submitting problem)
if(data["wtxType"]["name"]=="ln_deposit"){
txData = data["lnTx"]
txId = txData["id"]
isSettled = txData["settled"] // 1 for true
numSatsPaid = txData["num_satoshis"]
paymentRequest = txData["payment_request"]
helpers.log("bitcoin deposited: "+numSatsPaid)
if(isSettled == 1){
helpers.log("Publishing problem..")
// publish the problem because the payment is complete
helpers.publish_problem(res, txId, db)
}
}else if(data["wtxType"]["name"]=="ln_withdrawal"){
txData = data["lnTx"]
txId = txData["id"]
isSettled = txData["settled"] // 1 for true
numSatsPaid = txData["num_satoshis"]
paymentRequest = txData["payment_request"]
userName = data['passThru']['username']
helpers.log("bitcoin withdrawn: "+numSatsPaid)
if(isSettled == 1){
// withdrawl complete - update user's balance in database
users = db.collection('users')
users.find( {'username': userName} ).toArray( (err, user)=>{
user = user[0]
balance = user['balance'] - numSatsPaid
users.updateOne( {'username': userName}, {$set: {'balance': balance}})
} )
//users.updateOne( {'ln_txid': }, {$set: {'answers': answers}} )
}
res.send("success")
}else{
res.send("OK")
}
}) // app.post('/webhook')
Now every time there's payment, LNPAY will inform it to
/webhook endpoint of our server. You can see we're using data like txid to know which transaction the alert is about.What have we achieved by now? Let's pause and see. We've setup a method for our users to pay us, implemented a way that periodically checks if payment is complete on client side, and implemented a way by which our server knows a payment is completed.
This is good time to pause and realize we've implement a payment system for our website that works on an unprecedented scale. Anyone in the world can take part. A completely open and free system.
At this point users are able to post a question to your website with a bounty. Other people see and answer the questions you posted. You, as the problem poster, can accept the answer which solved the problem for you. I would spare you the details of how that happens. You can always see that in the source code available on github. These rewards are stored in a database and aren't actually paid out yet.
To cash out the rewards the user earned they go to their profile page and click on "Withdraw". This creates a pop up window with another QR code. Scanning this QR code initiates a withdraw request and the user is paid what they're owed.
There are 2 ways of withdrawing:
1) Withdraw using invoice
2) lnurl-withdraw
Just like we created invoices for users to pay us, users can also submit their invoice which you can pay to. In lnsolve.com I implemented lnurl-withdraw so I'll explain that here. Though you can use both if you wish.
lnurl-withdraw is a recent standard that makes withdrawing lightning bitcoin much more easier. It's a UX improvement from traditional withdrawal. Traditionally, the user has to generate an invoice from their wallet (which'll probably be on their phone) that they'll have to paste onto your website (which may not be on their phone).There's also a concern that the client can be compromised and the invoices can get swapped to that of the hacker when pasting.
lnurl-withdraw is more secure and user friendly way to withdraw.
You can read a complete overview on lnurl improvements here: https://degreesofzero.com/article/beyond-coffee-bitcoin-for-every-day-use-with-lnurl.html
lnurl-withdraw is implemented by requesting LNPAY a lnurl and then showing it as a QR code. The request is sent from client's browser to our server, which then checks if user owns the requested amount. If yes, server sends a request to LNPAY then returns lnurl data to the client.
Our server side end point looks like this:
app.get('/lnurl_withdraw', function(req, res){
if(!req.session.userAuthenticated){
res.send('not authenticated')
return
}
if(req.session.userName != req.query.username){
res.send(403)
return
}
userName = req.query.username // we can also use req.session.userName
amt = req.query.amt
// find user in db and check balance
users = db.collection('users')
users.find( {"username": userName} ).toArray( (err, user) => {
user = user[0]
balance = user['balance']
// if amt requested is within user's balance continue
if(balance >= amt){
memo = "lnsolve withdrawl for "+user['username']
helpers.log("lnurl-withdraw requested")
// sending request to lnpay to get lnurl-withdraw for the amount from our wallet
helpers.lnurl_withdraw(res, amt, memo, userName)
}else{
res.send('not enough balance')
return
}
}) // users
}) // app.get('lnurl_withdraw')
Here
helpers.lnurl_withdraw() is defined as this:var request = require('request');
var atob = require('atob');
var btoa = require('btoa');
module.exports = {
lnurl_withdraw: function(lnsolve_res, amt, memo, userName){
// according to doc: https://docs.lnpay.co/wallet/lnurl-withdraw
// passThru is base64 encoded json of data to use in webhooks, etc
passThruData = btoa(JSON.stringify({username: userName})) // reverse by atob()
var options = {
url: 'https://lnpay.co/v1/wallet/waka_a8BJX8vo8ax4RSYbF2Y7KIbO/lnurl/withdraw?passThru='+passThruData+'&num_satoshis='+amt+'&memo='+memo,
auth: {
'user': 'sak_...', // Your secret API Key here
'pass': ''
}
};
function callback(error, response, body) {
// body contains the lnurl
lnsolve_res.send(body)
}
request(options, callback);
}
}
Look at the
passThru value. When this payment is successful LNPAY will trigger an event and send POST request to webhook like earlier. passThru value is a value that is passed along throughout the entire webhook process.passThru is base64 encoded JSON of data. We use
btoa() to convert our data JSON to base64 and pass it as a parameter in URL. This JSON is then received by our webhook endpoint. We're passing the username in passThru so when webhook event is triggered our server will know which user's balance to update in database.btoa() is reverse of atob(). These aren't available in node.js by default. You can install them with npm i atob btoaYou can see the webhook code in case of withdrawals in section 4.4
Congratulations on reading till end!! You now have learnt the basics of making a lightning app. With these tools in your arsenal you can build many things. You can take the concepts you've learnt here and apply them to completely different platforms. You can implement these on a websites, apps, or even on smart fridges!
You can make more complex platforms which takes advantage of bitcoin's global scale.
Going forward, I'll post more posts like this (probably smaller) exploring more ways we can build lightning applications.
See ya on next post!