Skip to content

Commit

Permalink
added introduction to tutorial
Browse files Browse the repository at this point in the history
  • Loading branch information
facundoolano committed Jan 25, 2018
1 parent b3d44b7 commit 361ed7c
Show file tree
Hide file tree
Showing 2 changed files with 87 additions and 18 deletions.
105 changes: 87 additions & 18 deletions TUOTRIAL.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,87 @@
# Using riak_core in 2018

## Background

- explain what is riak core and why it's useful
- mention basho in receivership, current
- mention that documetation is scarce, include useful links: basho
where to start, rc wiki,
trytrytry, understanding rc, little rc book, elixir tutorial, slides links.
- explain the rationale for this tutorial: little rc book and rebar3
template have a lot of dependencies and doesn't work with E20;
closer to elixir tutorial but for erlang; document details we've
learned along the way
- explain the general structure is directly borrowed from those two
tutorials and will link to them when appropriate
# Using Riak Core in 2018

[Riak Core](https://github.com/basho/riak_core) is the distributed
systems framework used by the [Riak data store](https://github.com/basho/riak)
to distribute data and scale. More generally, it can be thought of as
a toolkit for building distributed, scalable, fault-tolerant
applications. What makes Riak Core so interesting and useful is that it implements
the ideas of the
[Amazon's Dyamo](https://en.wikipedia.org/wiki/Dynamo_(storage_system))
architecture and exposes its infrastructure as
a reusable library, allowing to easily apply them in any context that
can benefit from decentralized distribution of work (including but not
limited to data stores).

In practical terms, Riak Core is an Erlang/OTP application, and most
of the user defined work is done in the `riak_core_vnode` behavior.

## About this tutorial

Basho, the company that originally developed Riak and Riak Core was
put into receivership in 2017. This introduces some uncertainty about the
future of these products, although the community has shown interest in
taking over their maintenance. At the moment of writing,
that [riak_core_ng](https://github.com/Kyorai/riak_core) fork seems to
be the most actively maintained fork of Riak Core and hopefully the work being
done there will eventually be merged back to the canonical repository.

As part of our interest in this technology and our intention to use it
in new projects we had to struggle a bit with scarce and outdated
documenatation, stale dependencies, etc. The intention is thus to
provide a tutorial on how to use Riak Core today, on an Erlang 20
and rebar3 project, with minimal dependencies and operational
sugar. You'll notice the structure borrows heavily from
the
[Little Riak Core Book](https://marianoguerra.github.io/little-riak-core-book/)
and the
[riak_core in Elixir](https://medium.com/@GPad/create-a-riak-core-application-in-elixir-part-1-41354c1f26c3)
series, which were our main references.

## Useful links
* [Introducing Riak Core](http://basho.com/posts/business/introducing-riak-core/)
* [Riak Core Wiki](https://github.com/basho/riak_core/wiki)
* [Masterless Distributed Computing with Riak Core](http://www.erlang-factory.com/upload/presentations/294/MasterlessDistributedComputingwithRiakCore-RKlophaus.pdf)
* Ryan Zezeski's "working" blog:
[First, multinode](https://github.com/rzezeski/try-try-try/tree/master/2011/riak-core-first-multinode) and
[The vnode](https://github.com/rzezeski/try-try-try/tree/master/2011/riak-core-the-vnode)
* [Little Riak Core Book](https://marianoguerra.github.io/little-riak-core-book/)
* riak_core in Elixir:
[Part I](https://medium.com/@GPad/create-a-riak-core-application-in-elixir-part-1-41354c1f26c3),
[Part II](https://medium.com/@GPad/create-a-riak-core-application-in-elixir-part-2-88bdec73f368),
[Part III](https://medium.com/@GPad/create-a-riak-core-application-in-elixir-part-3-8bac36632be0),
[Part IV](https://medium.com/@GPad/create-a-riak-core-application-in-elixir-part-4-728512ece224) and
[Part V](https://medium.com/@GPad/create-a-riak-core-application-in-elixir-part-5-86cd9d2c6b92)
* [A Gentle Introduction to Riak Core](http://efcasado.github.io/riak-core_intro/)
* Understanding Riak Core:
[Handoff](http://basho.com/posts/technical/understanding-riak_core-handoff/),
[The visit fun](http://basho.com/posts/technical/understanding-riak_core-visitfun/) and
[Building Handoff](http://basho.com/posts/technical/understanding-riak_core-building-handoff/)

## 0. Riak Core overview

Riak Core is based on the Dynamo architecture, meaning it
scales and distributes the work in a decentralized manner, using
[Consistent Hashing](https://en.wikipedia.org/wiki/Consistent_hashing).

Most operations are applied to an object which is identified by some
value. In the context of a Key/Value store, for example, the
identifier is the Key used in get, put and delete operations.

Before performing the operation, a hashing function is applied to
the key. The hashed value will be used to decide what node of the
cluster should be responsible of executing the operation. The range of
all possible values the hashed key can take (the keyspace, usually
depicted as a ring), is partitioned in equally sized buckets, which
are called virtual nodes (vnodes).

![The Ring](ring.png)

The amount of vnodes is fixed and a given hash value will always
belong to the same partition (i.e. the same vnode). The vnodes, in
turn, are evenly distributed across all available physycal
nodes. Note this distribution isn't fixed as the keyspace partitioning
is: the vnode distribution can change if a physical node is added
to the cluster or a one goes down.

## 1. setup

Expand Down Expand Up @@ -43,17 +112,17 @@ put, get, delete

TODO #7

## 5. coverage commands
## 6. coverage commands

keys, values
explain the gotcha of breaking change in the coverage fsm

add a test

## 6. handoff
## 7. handoff

explain what we've learned digging the code comments

## 7. redundancy and fault tolerance
## 8. redundancy and fault tolerance

TODO #6
Binary file added ring.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 361ed7c

Please sign in to comment.