| Date post: | 22-Jan-2018 |
| Category: |
Technology |
| Upload: | scylladb |
| View: | 1,099 times |
| Download: | 0 times |
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smfthe fastest RPC fmwrk. in the west
Principal Engineer, Platform Engineering - Akamai Technologies
Alexander Gallego
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Alexander Gallego
2
Principal Engineer @ Akamai - Platform Group
Ex CTO / founder of Concord.io - A distributed
stream processor written in C++ atop Apache
Mesos (Now part of Akamai)
First employee, engineer for Yieldmo.com (ad-tech)
startup in NYC
maintainer of smf: github.com/senior7515/smf
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
background
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Can we do transactional streaming?
▪ At Concord.io, I worked on a streaming platform
o Can we do transactional writes (3x replication - even if in memory)
• Can we do it with low latency and high throughput?
– double digit ms *tail* latency at 1024 batches?
▪ Fastest open source queue did 150ms p90 and 2secs p9999
o Unpredictable JVM spikes -
• Spark once stalled for 4 seconds reading from Kafka - couldn’t replicate.
o Concord was in C++ - we wanted predictability
4
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Can we do better?
5
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
How about this! - per node overhead
6
7us p90 latency
26us p100 latency8us p99 latency
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
How about this! - per node overhead
7
Read Socket RPC Parsing
Method ExecutionFlush Socket
60 byte payload + 20 bytes of TCP - with full type serialization! p90 = 7 microseconds, p99 = 8 microseconds, p100 = 26 microseconds
p99=8us
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf RPC
▪ Built for microsecond tail latencies
▪ Atop seastar::future<>s
▪ IDL Compiler/CodeGen - using Google Flatbuffers’ IDL
▪ Multi-language compatibility
▪ Small - 16 byte overhead (with rich types, headers, compression,etc)
… it’s like gRPC / Thrift / Cap n’ Proto - for microsecond latencies.
9
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
IDL
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
11
namespace smf_gen.demo;
table Request { name: string;}table Response { name: string;}rpc_service SmfStorage { Get(Request):Response;}
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
12
namespace smf_gen.demo;
table Request { name: string;}table Response { name: string;}rpc_service SmfStorage { Get(Request):Response;}
input
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
13
namespace smf_gen.demo;
table Request { name: string;}table Response { name: string;}rpc_service SmfStorage { Get(Request):Response;}
output
input
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
14
namespace smf_gen.demo;
table Request { name: string;}table Response { name: string;}rpc_service SmfStorage { Get(Request):Response;}
Service Definition
output
input
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
15
namespace smf_gen.demo;
table Request { name: string;}table Response { name: string;}rpc_service SmfStorage { Get(Request):Response;}
smf_gen --filename demo_service.fbs
Service Definition
output
input
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Demo
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf::rpc_client
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
18
smf::rpc_typed_envelope<Request> req;req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);client->Get(req.serialize_data()).then([ ](auto reply) { std::cout << reply->name() << std::endl;});
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
19
smf::rpc_typed_envelope<Request> req;req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);client->Get(req.serialize_data()).then([ ](auto reply) { std::cout << reply->name() << std::endl;});
data to send
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
20
smf::rpc_typed_envelope<Request> req;req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);client->Get(req.serialize_data()).then([ ](auto reply) { std::cout << reply->name() << std::endl;});
data to send
actual socketseastar::shared_ptr<T>non-thread safe
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
21
smf::rpc_typed_envelope<Request> req;req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);client->Get(req.serialize_data()).then([ ](auto reply) { std::cout << reply->name() << std::endl;});
data to send
actual socketseastar::shared_ptr<T>non-thread safe
Method to call
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf::rpc_server
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
23
class storage_service : public SmfStorage { virtual seastar::future<rpc_typed_envelope<Response>> Get(rpc_recv_typed_context<Request> rec) final { rpc_typed_envelope<Response> data; data.data->name = "Hello, cruel world!"; data.envelope.set_status(200); return make_ready_future<decltype(data)>(std::move(data)); }};
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
24
class storage_service : public SmfStorage { virtual seastar::future<rpc_typed_envelope<Response>> Get(rpc_recv_typed_context<Request> rec) final { rpc_typed_envelope<Response> data; data.data->name = "Hello, cruel world!"; data.envelope.set_status(200); return make_ready_future<decltype(data)>(std::move(data)); }};
code-gen’ed service
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
25
class storage_service : public SmfStorage { virtual seastar::future<rpc_typed_envelope<Response>> Get(rpc_recv_typed_context<Request> rec) final { rpc_typed_envelope<Response> data; data.data->name = "Hello, cruel world!"; data.envelope.set_status(200); return make_ready_future<decltype(data)>(std::move(data)); }};
code-gen’ed service
Method
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
26
class storage_service : public SmfStorage { virtual seastar::future<rpc_typed_envelope<Response>> Get(rpc_recv_typed_context<Request> rec) final { rpc_typed_envelope<Response> data; data.data->name = "Hello, cruel world!"; data.envelope.set_status(200); return make_ready_future<decltype(data)>(std::move(data)); }};
code-gen’ed service
Method
return data
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf::rpc_filter
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
28
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
29
template <typename T>
struct rpc_filter {
seastar::future<T> operator()(T t);
};
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
30
struct zstd_compression_filter : rpc_filter<rpc_envelope> {
explicit zstd_compression_filter(uint32_t min_size)
: min_compression_size(min_size) {}
seastar::future<rpc_envelope> operator()(rpc_envelope &&e);
const uint32_t min_compression_size;
};
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
31
// add it to your clients
client->outgoing_filters().push_back(
smf::zstd_compression_filter(1000));
// add it to your servers
using zstd_t = smf::zstd_compression_filter;
return rpc.invoke_on_all(
&smf::rpc_server::register_outgoming_filter<zstd_t>,1000);
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf
32
static thread_local auto incoming_stage =
seastar::make_execution_stage("smf::incoming",
&rpc_client::apply_incoming_filters);
static thread_local auto outgoing_stage =
seastar::make_execution_stage("smf::outgoing",
&rpc_client::apply_outgoing_filters);
SE
DA
-pip
elin
ed
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
request anatomy
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf request anatomy
34
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf request anatomy
35
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
- Turn off padding by compiler.- Enforce layout. - Store everything in little endian- X-lang, X-platform compat- noop on most platforms
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf request anatomy
36
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf request anatomy
37
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf request anatomy
38
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?
max # of concurrent requests per client
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf request anatomy
39
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?
max # of concurrent requests per client
xxhash32 - very fast! 5.4GB/s
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf request anatomy
40
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?
max # of concurrent requests per client
xxhash32 - very fast! 5.4GB/s
request_id or status (response) code
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf Code Gen’d XOR id
41
auto fqn = fully_qualified_name;
service_id = hash( fqn(service_name) )
method_id = hash( ∀ fqn(x) input_args_types,
∀ fqn(x) output_args_types,
fqn(method_name),
separator = “:”)
rpc_dispatch_id = service_id ^ method_id;
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf Code Gen’d XOR id
42
/// RequestID: 212494116 ^ 1719559449
/// ServiceID: 212494116
/// MethodID: 1719559449
future<smf::rpc_recv_typed_context<Response>>
Get(smf::rpc_envelope e) {
e.set_request_id(212494116, 1719559449);
return send<smf_gen::demo::Response>(std::move(e));
}
Method ID
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf Code Gen’d XOR id
43
handles.emplace_back(
"Get", 1719559449,
[this](smf::rpc_recv_context c) {
using req_t = smf::rpc_recv_typed_context<Request>;
auto session_id = c.session();
return Get(req_t(std::move(c))).then(
[session_id](auto typed_env){
typed_env....mutate_session(session_id);
return make_ready_future<rpc_envelope>(
typed_env.serialize_data());
});
Method ID
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf Code Gen’d XOR id
44
struct rpc_service {
virtual const char *service_name() const = 0;
virtual uint32_t service_id() const = 0;
virtual std::vector<rpc_service_method_handle> methods() = 0;
virtual ~rpc_service() {}
};
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
telemetry
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf built in telemetry
46
High Dynamic Range Histogram (HDR) … Expensive 185 KB
::hdr_init(1, // 1 microsec - minimum value
INT64_C(3600000000), // 1 hour in microsecs - max value
3, // Number of significant figures
&hist); // Pointer to initialize
// clients
client = ClientService::make_shared(std::move(opts));
client->enable_histogram_metrics();
// servers enabled by default
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf built in telemetry (prometheus)
47
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
performance?
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf DPDK client - DPDK server*
49
7us p90 latency
26us p100 latency8us p99 latency
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf end-to-end latency (DPDK)
50
p100=500usp90=51usp99=56us
2 Threads. Includes connection open time - cold cache
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf end-to-end latency (DPDK)
51
Same graph, minus the **first** request of each of the 2 threads
p100=151us
p50=51us p99=56us
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf future work
52
● Currently could only do 1.5MM qps on the server setup ○ size = 60 byte payload + 20 TCP frame bytes○ Hit TCP.hh bug in seastar with httpd/seawreck and my own impl
■ `(_snd.window > 0) || ((_snd.window == 0) && (len == 1))' failed.
■ Could be my lab setup■ Because of this - couldn't fill the wire fast enough
● Add JVM, Python, Go, codegen
● Improve Docs: https://senior7515.github.io/smf/
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
THANK YOU
[email protected] | alexgallego.org
@emaxerrno
Please stay in touch
Any questions? https://senior7515.github.io/smf/
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
extra
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
smf Write Ahead Log (latency)
55
percentile Apache Kafka smf WAL speedup
p50 878ms 21ms 41X
p95 1340ms 36ms 37x
p99 1814ms 49ms 37x
p999 1896ms 54ms 35x
p100 1930ms 54ms 35x
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
56
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
