NATS and JetStream

Intro

• Scalable and resilient message broker
• Written in Go programming language
  • Initial version written in Ruby (Derek Collison)
  • Then rewritted into Go
    • just one executable
    • started much faster than java -version :)
• Based on multiple cooperating components
• No single point of failure

NATS components

• Server
  • gnatsd previously
  • nats-server today
• Interfaces for various ecosystems
• Streaming support
  • NATS Streaming Server (previously)
  • JetStream as modern variant
• NATS Connector Framework

NATS ecosystems support

Official

1. C
2. C#
3. Elixir
4. Go
5. Java
6. NGINX
7. Node.js
8. Python Asyncio
9. Python Tornado
10. Ruby
11. TypeScript

Unofficial

1. .NET
2. Arduino
3. Clojure
4. Elm
5. Erlang
6. Haskell
7. Lua
8. MicroPython
9. PHP
10. Perl
11. Python
12. Python Twisted
13. Qt5 C++
14. Rust
15. Scala
16. Spring API
17. Swift

Communication strategies

• pub-sub
  • using subjects (similar to topics)
• push-pull (via queues)
• request-reply
  • point-to-point
  • one-to-many
  • (based on inbox for replies)
• streaming
  • with ability to “replay” messages
  • various retention and delivery policies
  • subscribers’ offsets stored on server
    • (no Kafka madness with ZooKeeper etc.)
• key/value store
• object store

Overview of communication strategies

pub-sub communication pattern

Multiple producers to given subject

push-pull communication pattern via queue

All consumers work concurrently with same messages

Round-robin approach / load balancing

request-reply communication pattern

Streaming: push-push communication pattern

Streaming: push-pull communication pattern

Quality of service

• At most once QoS
• At-least / exactly once QoS
• 1, 3, or 5 replicas
  • 2 and 4 possible too, but it does not make much sense

Protocol used by NATS

• text-based
  • but data can be binary
• possible to test it via telnet

Command   send by
INFO      server
CONNECT   client
PUB       client
SUB       client
UNSUB     client
MSG       server
PING      client and server
PONG      client and server
+OK       server
-ERR      server

Communicating with NATS

Live demo

cd /tmp/ramdisk/nats-0.0.34-linux-amd64/

./nats server run --jetstream

export PS1=$\ 

cd /tmp/ramdisk/nats-0.0.34-linux-amd64/

./nats stream create foo

./nats stream create bar

./nats stream rm foo

./nats stream rm bar

./nats stream ls

./nats stream create user-add-operation

./nats stream create user-delete-operation

./nats stream ls

./nats pub user.alice.add "Alice added"

./nats pub user.bob.add "Bob added too"

./nats pub user.clayton.delete "Clayton deleted"

./nats pub user.random.add "User #" --count=10

./nats sub --stream user-add-operation

./nats sub --stream user-add-operation --last

./nats sub --stream user-add-operation --new

Appendix

Links

• About NATS
• Slides about NATS
• NATS comparison

Czech articles about NATS

• Komunikace s message brokery z programovacího jazyka Go
• Použití message brokeru NATS
• NATS Streaming Server