Skip to content

Commit

Permalink
Feature/triage (#5)
Browse files Browse the repository at this point in the history
* wip

* wip

* wip

* pub/sub minimum almost working

* updated threads

* wip

* wip

* wip

* tested 100k+ set size and elastic

* wip

* wip

* wip

* wip

* wip

* wp

* wp

* wip

* added www

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip
  • Loading branch information
dioptre authored Mar 19, 2020
1 parent bb7f9dc commit 6d12d44
Show file tree
Hide file tree
Showing 2 changed files with 100 additions and 38 deletions.
136 changes: 99 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
# Messaging Exchange and Event Bus for Node.js & Python
# Messaging Exchange and Event Bus for Node.js & Reports in Python, PySpark

Subsystem of the SFPL growth and experimentation framework.

### 🔴 Important 🔴
```diff
- This project is a work in progress. Please come back shortly.
```
## Design

Request -> Prioritization -> Triage (write [messageid/dateuuid, owner]; [owner, msgs], [option Realtime, Nearline, scheduled, failed], [capture,reporting,recall])
**Essentially this is Optimizely for messaging**

Tracking messaging like we do the rest of internet traffic is essential to understanding our customers and optimizing growth. Messaging exchange (MSGXC) is a central system for dispatching and tracking all messgaing (currently supports iOS native, Android native, SMS, Email, Websockets, WebPush Notifications).

### Components

* Realtime
* Websockets (Socket.io, uWebSockets)
* Websockets
* Web Notifications
* Native Messaging
* Nearline
Expand All @@ -25,7 +29,7 @@ Request -> Prioritization -> Triage (write [messageid/dateuuid, owner]; [owner,
* Native messaging receipts
* Offline
* PySpark Reports
* Scheduler
* Scheduler


### Urgency
Expand All @@ -41,7 +45,9 @@ Request -> Prioritization -> Triage (write [messageid/dateuuid, owner]; [owner,

Run ```docker-compose up``` in the root of this project to get elassandra (cassandra with elastic search), and NATS.io working.

### API
## API

### User Functions

#### GETs

Expand All @@ -57,6 +63,91 @@ Checks whether the server is up.
```200```


#### POSTs

##### Enlist
_Requires Any JWT_
```
/api/v1/enlist
```

Enlists a user's devices including web-browsers, native android/os, to receive messages.

**Request body:**
```
{"os": "ios", "token": "ad62ea6ea23d6974871cf59a06cbdb2783b85adbafe3355c0007362249d3e75c"}
*or*
{"os": "android", "token": "e_nslPZejyM:APA91bHR-znf4EuSIKeY9dzlX4cupXA5cdsW1SzOHUFRrsteaL5WDuzsh_cnpVpQC3IPcewl_v3N0kbArC67UTEW_ENt5Ej5Sn0qi1RoRHv5beLNi9y4OzZ__T3SH3tW5gwqxn2Hap01"}
*or*
{
endpoint: 'https://fcm.googleapis.com/fcm/send/czVGxJOoycA:APA91bGZt8FZU2fEAnDCr1PdRb7HDtayoGDUO1dy6vjTu1sDKhGAAB0i2nXw_jGKhnzmh5rmK4klsyaRQaUpM0oS0VYGBCTTpF-nkP67UEW7BX9o7vNatcPuiG-yHa75hXE80B5F7DNi',
expirationTime: null,
keys: {
p256dh: 'BN81HfHxzt4V4lXDV_2ia8Rl_QvqofvoNI7_MOHCiOMEYuFXFNDBOJzKCCj2nzK5luwnH6rvBCN2jx7lNHpraaw',
auth: '0ywNrBxQ-rm4skkHDJB5lw'
}
}
```

**Correct Response:**
```true```

##### Publish
_Requires Any JWT_
```
/api/v1/publish
```

Publish a message to a thread (this sends a message to the mthread).

**Request body:**
```
{"tid": "5ae3c890-5e55-11ea-9283-4fa18a847130", "msg":"the message you want to send", "opts", { }}
```

**Correct Response:**
```true```


##### Subscribe
_Requires Any JWT_
```
/api/v1/subscribe
```

Subscribe a user to a thread (this adds a user to the subs column in mthreads).

**Request body:**
```
{"tid": "5ae3c890-5e55-11ea-9283-4fa18a847130"}
```

**Correct Response:**
```true```


##### Unubscribe
_Requires Any JWT_
```
/api/v1/unsubscribe
```

Unsubscribe a user from a thread (this removes a user from the subs column in mthreads).

**Request body:**
```
{"tid": "5ae3c890-5e55-11ea-9283-4fa18a847130"}
```

**Correct Response:**
```true```

### Admin Functions

#### POSTs

##### BROADCAST
Expand Down Expand Up @@ -96,7 +187,7 @@ opts _Optional_
```true```

##### SEND
_Requires Any JWT_
_Requires Admin JWT_
```
/api/v1/send
```
Expand All @@ -114,39 +205,10 @@ opts _Optional_
**Correct Response:**
```true```

##### SUBSCRIBE-NATIVE
_Requires Any JWT_
```
/api/v1/native/subscribe
```

Subscribes a user to native messaging using their current device.

**Request body:**
```
{"os": "ios", "token": "ad62ea6ea23d6974871cf59a06cbdb2783b85adbafe3355c0007362249d3e75c"}
or
{"os": "android", "token": "e_nslPZejyM:APA91bHR-znf4EuSIKeY9dzlX4cupXA5cdsW1SzOHUFRrsteaL5WDuzsh_cnpVpQC3IPcewl_v3N0kbArC67UTEW_ENt5Ej5Sn0qi1RoRHv5beLNi9y4OzZ__T3SH3tW5gwqxn2Hap01"}
or
{
endpoint: 'https://fcm.googleapis.com/fcm/send/czVGxJOoycA:APA91bGZt8FZU2fEAnDCr1PdRb7HDtayoGDUO1dy6vjTu1sDKhGAAB0i2nXw_jGKhnzmh5rmK4klsyaRQaUpM0oS0VYGBCTTpF-nkP67UEW7BX9o7vNatcPuiG-yHa75hXE80B5F7DNi',
expirationTime: null,
keys: {
p256dh: 'BN81HfHxzt4V4lXDV_2ia8Rl_QvqofvoNI7_MOHCiOMEYuFXFNDBOJzKCCj2nzK5luwnH6rvBCN2jx7lNHpraaw',
auth: '0ywNrBxQ-rm4skkHDJB5lw'
}
}
```

**Correct Response:**
```true```

## TODO

- [ ] Request -> Prioritization -> Triage (write [messageid/dateuuid, owner]; [owner, msgs], [option Realtime, Nearline, scheduled, failed], [tracking,capture,reporting,recall])
- [ ] Scheduler
- [ ] Processing in batches
- [ ] Multicast using elastic search (instead of slower CQL)
- [x] Add web notifications and SMS (Amazon/Twilio)
Expand Down
2 changes: 1 addition & 1 deletion xcs/pods/messaging/thread/model.js
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ class Threading {

//////////////////////////
//SEND OTHERS
//TODO: Triage
//TODO: Triage!
//TODO: Add retries for realtime
//TODO: ScheduleDegraded() and include nearline messaging emd, emm , emw (email summaries)
notified.push(thread.subs[i]);
Expand Down

0 comments on commit 6d12d44

Please sign in to comment.