Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
9a4dcf4
pres
komuw Jul 10, 2018
6a6f500
pres
komuw Jul 10, 2018
b8b8a3d
piych
komuw Jul 10, 2018
dba651f
piych
komuw Jul 10, 2018
890c2d3
piych
komuw Jul 10, 2018
aafc2c4
piych
komuw Jul 10, 2018
1805b69
piych
komuw Jul 10, 2018
ac18d1a
piych
komuw Jul 10, 2018
eb647cf
piych
komuw Jul 10, 2018
e1bf284
piych
komuw Jul 10, 2018
d2d8ad8
piych
komuw Jul 10, 2018
1c9b2b0
piych
komuw Jul 10, 2018
f6b8395
piych
komuw Jul 10, 2018
8eddcbc
piych
komuw Jul 10, 2018
e286dcf
piych
komuw Jul 10, 2018
913bae3
piych
komuw Jul 10, 2018
160c832
piych
komuw Jul 10, 2018
0c61cd7
piych
komuw Jul 10, 2018
4c5361d
piych
komuw Jul 10, 2018
92529cb
piych
komuw Jul 10, 2018
f3e784d
piych
komuw Jul 10, 2018
a6b3bdb
piych
komuw Jul 10, 2018
0f5b3ec
piych
komuw Jul 10, 2018
cb7d5ff
piych
komuw Jul 10, 2018
f299903
piych
komuw Jul 10, 2018
04a0788
piych
komuw Jul 10, 2018
264d17c
piych
komuw Jul 10, 2018
4f3dfef
piych
komuw Jul 10, 2018
6434dab
piych
komuw Jul 10, 2018
5dd07cc
piych
komuw Jul 10, 2018
0e4cf3f
piych
komuw Jul 10, 2018
8302d26
piych
komuw Jul 11, 2018
0a26cb0
update
komuw Oct 22, 2018
76210f9
update
komuw Oct 22, 2018
6b4cd6b
update
komuw Oct 22, 2018
d0be280
update
komuw Oct 22, 2018
018cd74
update
komuw Oct 22, 2018
fa46717
update
komuw Oct 22, 2018
c622eb8
update
komuw Oct 22, 2018
3a62d7c
Merge branch 'master' into pres
komuw Oct 23, 2018
2873d16
update
komuw Oct 23, 2018
444fa8f
update
komuw Oct 23, 2018
50b855f
update
komuw Oct 23, 2018
0810702
update
komuw Oct 23, 2018
692b2d0
update
komuw Oct 23, 2018
b360ad9
update
komuw Oct 23, 2018
cb9a013
update
komuw Oct 23, 2018
96eaf1d
update
komuw Oct 23, 2018
13ebe23
update
komuw Oct 23, 2018
8dbe255
update
komuw Oct 23, 2018
2f943c5
update
komuw Oct 23, 2018
5fdb97a
update
komuw Oct 23, 2018
d8e3faf
update
komuw Oct 23, 2018
b7f9d68
update
komuw Oct 23, 2018
276e267
update
komuw Oct 23, 2018
c0c1d52
update
komuw Oct 23, 2018
f074f65
update
komuw Oct 23, 2018
68056be
update
komuw Oct 23, 2018
01f0068
update
komuw Oct 23, 2018
0ea7c85
update
komuw Oct 23, 2018
8cf4c0d
update
komuw Oct 23, 2018
6fb94a7
update
komuw Oct 23, 2018
34a5e99
update
komuw Oct 23, 2018
21f8017
update
komuw Oct 23, 2018
a27a7db
update
komuw Oct 24, 2018
e4f9bcb
update
komuw Oct 24, 2018
218cb76
update
komuw Oct 24, 2018
bfb82ef
update
komuw Oct 24, 2018
dec4791
update
komuw Oct 24, 2018
484519f
update
komuw Oct 24, 2018
b95e23f
update
komuw Oct 24, 2018
7655f96
update
komuw Oct 24, 2018
180f48a
update
komuw Oct 24, 2018
d559db4
update
komuw Oct 24, 2018
20e10c4
update
komuw Oct 24, 2018
00658b3
update
komuw Oct 24, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,32 @@ test:
@printf "\n run flake8::\n" && flake8 .
@printf "\n run pylint::\n" && pylint --enable=E --disable=W,R,C --unsafe-load-any-extension=y examples/ naz/ tests/ cli/
@printf "\n run bandit::\n" && bandit -r --exclude .venv -ll .
@printf "\n run mypy::\n" && mypy --show-column-numbers -m naz.q -m naz.throttle -m naz.ratelimiter -m naz.hooks -m naz.sequence
@printf "\n run mypy::\n" && mypy --show-column-numbers -m naz.q -m naz.throttle -m naz.ratelimiter -m naz.hooks -m naz.sequence


asciinema rec --idle-time-limit 2 first.cast
asciinema play -i 2 docs/first.cast
asciinema play -i 2 -s 2 docs/first.cast


---
#### 1.2.1 codeee
![codeee](docs/2times.png)


---
#### 1.2.2 codee2
![codeee](docs/1times.png)


---
![codeee](docs/2times.png) {this is what we will do}


---
![codeee](docs/1times.png)


Use https://carbon.now.sh for code png's


217 changes: 217 additions & 0 deletions PITCHME.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
---
## Introduction to naz, an async SMPP client

https://gitpitch.com/komuw/naz/presentation

---
#### topics
1. SMPP
2. SMPP & python intro
3. current lay of the land
4. naz intro
5. naz features
6. Q&A


---
#### 1. intro
**Name:** Komu Wairagu
**Occupation:** Software developer at [https://jumo.world/](https://jumo.world/)

**About Me:** https://www.komu.engineer/about

---
#### 1.1 SMPP
Short Message Peer-to-Peer.
It's a protocol designed for transfer of Short messages between an SMS server and a mobile phone.
Based on exchange of request/response protocol data units(PDUs) between client & server over TCP/IP network.


---
#### 1.2 Why care about SMPP?
Typically used for SMS and USSD by Telcos(Mobile Network Operators).
If you want to do integrate with various Telcos; you'll have to SMPP.


---
#### 1.3 sequence of requests
![Image of sequence](docs/pyconKE2018/request-response-sequence.png)


---
#### 1.4 PDU format
![Image of pdu format](docs/pyconKE2018/pdu-format.png)


---
#### 2. SMPP & python
How do you connect to SMPP server(SMSC) from Python?


---
![SMPP & python](docs/pyconKE2018/python-smpp-intro.png)


---
#### 3. current lay of the land
- github.com/podshumok/python-smpplib
- github.com/praekelt/vumi
- ... couple more

---
#### 3.1 problems with current solutions
- complexity of code base
- coupling with other things(rabbitMQ, redis, Twisted)
- non-granular configurability
- (you can only set `throttle_delay: X seconds` )
- maintenance debt:
- (we had to disable vumi sentry integration;
vumi outdated raven dependancies)
- cant migrate your app to Python3(because vumi is py2)
- lack of visibility(what is happenning?)
- (you have to enrich vumi logs).


---
#### 4. naz intro
naz is an async SMPP client.
It's easily configurable, BYO(throttlers, rateLimiters etc)

```bash
pip install naz
```

---
#### 4.1 architecture
| Your App | ---> | Queue | ---> | Naz |
what is the Queue?? inMem, rabbitmq, redis ...??
Naz makes no imposition of what the Queue is.
BYO queue...


---
![naz exampleusage](docs/pyconKE2018/naz-example-usage.png)


---
#### 4.2.1 sequence of requests
![Image of sequence](docs/pyconKE2018/request-response-sequence.png)


---
#### 5. naz features
running theme: configurability, observability, BYO ... nini nini


---
#### 5.1.1 observability: logging
![naz-observability-logging](docs/pyconKE2018/naz-observability-logging.png)

---
#### 5.1.1 observability: logs
![naz-observability-logs](docs/pyconKE2018/naz-observability-logs.png)



---
#### 5.1.2 observability: hooks
An instance of a class that implements `naz.hooks.BaseHook`. It has two methods `request` and `response`.
create an instance implementation of `BaseHook`, plug it in, and u can do whatever u want inside `request`/`response` methods.

---
![naz-observability-hooks-example1](docs/pyconKE2018/naz-observability-hooks.png)


---
#### 5.1.2 observability: hooks example 2
![naz-observability-hooks-prometheus](docs/pyconKE2018/naz-observability-hooks-prometheus.png)


---
#### 5.2 Rate limiting
An instance of a class that implements `naz.ratelimiter.BaseRateLimiter`. It has one method `limit`.
create an instance implementation of `BaseRateLimiter`, plug it in, and u can implement any rate limiting algo inside `limit` method.
`naz` ships with a simple token-bucket Ratelimiter, `SimpleRateLimiter`


---
#### 5.2 Rate limiting: example
![naz-ratelimiting-simple](docs/pyconKE2018/naz-ratelimiting-simple.png)


---
#### 5.2 Rate limiting - logs
![naz-ratelimiting-simple-logs](docs/pyconKE2018/naz-ratelimiting-simple-logs.png)


---
#### 5.2 Rate limiting: example2
![naz-ratelimiting-awesome](docs/pyconKE2018/naz-ratelimiting-awesome.png)



---
#### 5.4 Throttle handling
An instance of a class that implements `naz.throttle.BaseThrottleHandler`.
`naz` calls it to handle throttling events from Telco.
`naz` ships with a default, `SimpleThrottleHandler`

---
#### 5.4 Throttle handling; example
![naz-throttle-handling](docs/pyconKE2018/naz-throttle-handling.png)


---
#### 5.5 Queuing
An instance of a class that implements `naz.q.BaseOutboundQueue`. It has two methods `enqueue` & `dequeue`.
what you put inside those two methods is upto you.
Your app queues messages, naz consumes from that queue and then sends those messages to SMSC/server.
`naz` ships with a `SimpleOutboundQueue` that queues in Memory.

---
#### 5.5 Queuing; example
![naz-redis-queue](docs/pyconKE2018/naz-redis-queue.png)



---
#### 5.5 Queuing; example (your app)
![naz-redis-queue-app](docs/pyconKE2018/naz-redis-queue-app.png)

---
#### 5.6 cli app
this is installed when you install `naz`.
```sh
naz-cli --help
```
```bash
usage: naz [-h] [--version] --config CONFIG

naz is an SMPP client.
example usage: naz-cli --config /path/to/my_config.json

optional arguments:
-h, --help show this help message and exit
--version The currently installed naz version.
--config CONFIG The config file to use. eg: --config
/path/to/my_config.json
```

---
#### 5.6 cli app; example
demo


---
#### 6. resources
- https://github.com/komuw/naz
- https://github.com/komuw/naz/blob/master/docs/SMPP_v3_4_specification.pdf
- https://gitpitch.com/komuw/naz/presentation
- https://github.com/praekelt/vumi

- https://www.komu.engineer/about


---
#### 6. Thanks
Q & A
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ naz is in active development and it's API may change in backward incompatible wa
+ [Throttle handling](#4-throttle-handling)
+ [Queuing](#5-queuing)

https://gitpitch.com/komuw/naz/presentation

## Installation

Expand Down
Binary file added docs/pyconKE2018/naz-example-usage.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
26 changes: 26 additions & 0 deletions docs/pyconKE2018/naz-example-usage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import naz, asyncio

loop = asyncio.get_event_loop()
outboundqueue = naz.q.SimpleOutboundQueue(maxsize=1000, loop=loop)
cli = naz.Client(
async_loop=loop,
smsc_host="127.0.0.1",
smsc_port=2775,
system_id="smppclient1",
password="password",
outboundqueue=outboundqueue,
)

# 1. network connect and bind
reader, writer = loop.run_until_complete(cli.connect())
loop.run_until_complete(cli.tranceiver_bind())
try:
# 2. send SMS, read responses from SMSC, send status checks
tasks = asyncio.gather(cli.send_forever(), cli.receive_data(), cli.enquire_link())
loop.run_until_complete(tasks)
except Exception as e:
print("exception occured. error={0}".format(str(e)))
finally:
# 3. unbind
loop.run_until_complete(cli.unbind())
loop.close()
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pyconKE2018/naz-observability-hooks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
40 changes: 40 additions & 0 deletions docs/pyconKE2018/naz-observability-hooks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import sqlite3
import naz

class SetMessageStateHook(naz.hooks.BaseHook):
async def request(self, smpp_event, correlation_id):
pass
async def response(self, smpp_event, correlation_id):
if smpp_event == "deliver_sm":
conn = sqlite3.connect('mySmsDB.db')
c = conn.cursor()
t = (correlation_id,)
c.execute("UPDATE \
SmsTable \
SET State='delivered' \
WHERE CorrelatinID=?", t)
conn.commit()
conn.close()

stateHook = SetMessageStateHook()
cli = naz.Client(
...
hook=stateHook,
)

import naz
from prometheus_client import Counter

class MyPrometheusHook(naz.hooks.BaseHook):
async def request(self, smpp_event, correlation_id):
c = Counter('my_requests', 'Description of counter')
c.inc() # Increment by 1
async def response(self, smpp_event, correlation_id):
c = Counter('my_responses', 'Description of counter')
c.inc() # Increment by 1

myHook = MyPrometheusHook()
cli = naz.Client(
...
hook=myHook,
)
Binary file added docs/pyconKE2018/naz-observability-logging.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 7 additions & 0 deletions docs/pyconKE2018/naz-observability-logging.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import naz
cli = naz.Client(
...
log_metadata={
"env": "prod", "release": "canary", "work": "jira-2345"
}
)
Binary file added docs/pyconKE2018/naz-observability-logs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pyconKE2018/naz-ratelimiting-awesome.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pyconKE2018/naz-ratelimiting-simple.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
22 changes: 22 additions & 0 deletions docs/pyconKE2018/naz-ratelimiting.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import naz
limiter = naz.ratelimiter.SimpleRateLimiter(
send_rate=1, max_tokens=1, delay_for_tokens=6
)
cli = naz.Client(
...
rateLimiter=limiter,
)


import naz
class AwesomeLimiter(naz.ratelimiter.BaseRateLimiter):
async def limit(self):
sleeper = 13.13
print("\n\t rate limiting. sleep={}".format(sleeper))
await asyncio.sleep(sleeper)

lim = AwesomeLimiter()
cli = naz.Client(
...
rateLimiter=lim,
)
Binary file added docs/pyconKE2018/naz-redis-queue-app.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pyconKE2018/naz-redis-queue.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading