Tech Stacks

Live Streaming CDN | PRJ.LiveStreaming (AWS Live Streaming (cost/hr)

• $151/hr/1,000 viewers, calculated from …
  $753/hr/5,000 viewers …
  • AWS Elemental MediaLive (transcoding)
    $4.87 + $1.176 = $6.046/hr
  • CloudFront CDN (distribution)
    $747.07/hr (99% of the cost)
    

Wowza OVP

• $96.00/hr/1000 viewers …
  • 3 TB @ $200/mo for 3 TB 15GB/$1 ($0.067/GB)
    @ 1 hr @ 400 KB/s = 1.44 GB
    x 1,000 viewers = 1.44 TB
    $96.00/hr (of the $200/mo alottment)
• uscreen.tv (Now Unauthorized.TV ?)
• $400/mo for 2,500 viewers (1,000 concurrent)

YouTube

• Bitrate @ 720p playback 230 KB/s (1.8 Mbps)
• Bitrate @ 720p playback 325 KB/s (2.6 Mbps)

Unauthorized.tv

• Bitrate @ 720p playback 460 KB/s (3.7 Mbps)
• DevTools > Network > Req/Resp while playing video (<iframe src="https://js.stripe.com/v2/m/outer.html#url=https://www.unauthorized.TV ..." ...> </iframe> ... <iframe src="https://m.stripe.network/inner.html#url=https://www.unauthorized.TV ..." ...> </iframe>

Slack | @Wikipedia

Searchable Log of All Conversation and Knowledge (SLACK)

• Group/team messaging/voice/video. Searchable record.
• Tech Stack; custom protocol; originally IRC;
  public API; 3rd-party integrations.
  
  • Backend
    • AWS servers @ Kubernetes; Terraform, Chef.
    • Chat (msgs) @ Java+Go; WebSockets
    • Voice + video @ Elixr; some Node.js
    • gRPC, Thrift, JSON-over-HTTP
      
    • PHP/Hacklang @ HHVM
    • Kafka and Redis async queue system.
      
    • MySQL/Vitess/Memcached+MCRouter
      
    • SolrCloud search
    • HAproxy LB; Consul for config + service discovery.
      
    • Data warehouse @ Presto, Spark, Airflow, Hadoop and Kafka.
      
    • Metrics @ Prometheus; logging @ ELK.
  • Frontend (Web; Desktop; Android; iOS)
    • React/ES6; Electron; Java/Kotlin; C/Swift
      
• Freemium Model
  
  • Free to search up to 10,000 archived messages.
    
  • Paid for unlimited, and add unlimited apps and integrations.
• Biz Growth
  
  • 2013: 8,000 signups in 24 hrs of launch.
    2015: 10,000 new DAU/wk; 135,000 paying customers; 60,000 teams.
    2015: 200,000 paid subs; 750,000 DAU.
    2018: 8M active users; 3M paid accounts.
    2019: 10M DAU; 600,000 orgs; 150 countries.

WhatsApp | @Wikipedia | @quora.com | @highscalability.com

• Messaging and VoIP service
  • IM originally; +voice @ 2015; +video @ 2016; IM-delete(@<7min) @ 2017
    
  • Multimedia img/audio/vid messages are uploaded to HTTP server; server sends content link (incl. Base64-encoded thumbnail if img|vid) to app.
    
  • End-to-end Encryption
    
  • WhatsApp Payments (P2P)
    
    • @ India only; in-app payments and money transfers using Unified Payments Interface (UPI); enables account-to-account transfers from a mobile app without having any details of the beneficiary's bank.
  • Cryptocurrency project @ 2019.
    
• Erlang @ FreeBSD; XMPP (customized version); Mnesia DB.
  
  • Ejabberg server; Jabber/XMPP + HTML5 WebSockets.
    
    • @ 2019, 1.5B users; @ 2014, 10 billion messages a day @ 500M Active Users
    • Message queue length is primary gauge of system health.
      
    • Hot-loading means updates/fixes can be pushed without restarts or traffic shifting; loosely-coupled; easy to roll changes out incrementally.
      
      • SSL socket to the WhatsApp server pools.
        
      • All messages are queued on server until client reconnects/retrieves.
        
      • Messages are wiped from the server memory on client receipt.
        
        • Retrieval notice sent back to server; forwards this status back to sender. (Sender sees "checkmark" icon next to message).
          
• Registering Users (Sign-up)
  User ID = Jabber ID = {phone-number}@s.whatsapp.net . Originally, created a user/pass based on user's phone IMEI number; changed recently. Now uses client request to send temporary 5 digit PIN via SMS to client phone. Using the PIN, client app requests/recieves/stores a unique key (password), used thereafter to validate.
  
  • Newer method, PIN per SMS, allows registering a client app on a different device.
    
  • The key is stored in the app/device; registering new device invalidates old key.

WeChat/Tencent QQ (IM) China | US

• Erlang @ OpenStack; Tencent Cloud (TStack).
  
  • Ejabberg server; XMPP + HTML5 WebSockets.
    
  • Client-side storage (EnMicroMsg.db) is an encrypted SQLite database (SQLcipher).
    
• Mini-Program (3rd-party) Devs | MINA API/framework | other frameworks | GitHub user Wiki | Search
  
  • MP (app) embeds in WeChat app; Javascript + WeChat API.
    
  • An MP can sell direct to consumers per WeChat payment services.
    
  • Open and use! No download, no install.
    
  • 1M apps (MPs) @ 2019; already half that of Apple Store;
    
    • 35% of MP traffic is sharing via chat (sends a sharing card).
      
• WeChat Pay
  
  • Funds transfer is not immediate;
    bank account linked; Visa, MasterCard & JCB (Discover/UnionPay/RuPay)
    
  • Digital Wallet; money transfers between contacts, and to 3rd parties;
    
  • Virtual Red Envelope; send money as gift; very popular
    
  • Alipay/Alibaba (Ant Financial) is competitor.
    
    • Funds transfer is not immediate;
      settlement time depends on the payment method;
      unlike Instant Pay systems, e.g., Venmo (PayPal Mobile), Zelle.
      
    • USA affiliate First Data
      

Viper/Rakuten

• IM/VoIP/video; developed in Israel.
  
• Users are registered and identified through a cellular telephone number.
  
• 1B registered users @ 2018
  Japan & Russia are its big markets.

Comments as a Service

Commento commenting platform | @GitHub

• Golang; "A fast, bloat-free, privacy-focused commenting platform"
  
• Commento, Inc. 2016, Delaware, #6797473
  adtac: Adhityaa Chandrasekar
  
• Show HN: Commento:
  "a fast, privacy-focused alternative to Disqus" (commento.io)
  

DISQUS | Disqus tech [2014]

• Python/Django (DISQUS) => Golang (Realtime) => 5x NGINX (Push Stream)
  
  • New Posts -> Disqus -> redis queue
    -> “python glue” Gevent formatting server (2 servers)
    -> http post -> nginx pub endpoint
    -> nginx + push stream module (5 servers)
    <- clients
    
    • 1M connections concurrently; 500M users @ 2014
      
    • 3,200 connections/sec
      
    • 150K/130K (TX/RX) packets
      
    • 150/80 Mbps (TX/RX)
      
    • < 15ms latency end-to-end; faster than JS front-end can render
      
• @ Python
  
  • Greenlets (gevent pkg); lightweight thread-like structure.
    
• @ NGINX | @YouTube
  
  • Ubuntu sysctl settings to handle max load
    
  • Shrink prealloc @ gzip on NGINX to 32KB; default is 264KB/conn; content is only ~ 2KB
    
    

JAMstack

• Lambda (JSON) -> SMS-slack (Approval) -> Lambda -> Static HTML Generator/Server

Facebook comments page

• 2 MB of files + 200 KB page (.html)