| Time |
S |
Nick |
Message |
| 01:28 |
|
|
semiosis joined #sourcefu |
| 11:46 |
|
|
onder` joined #sourcefu |
| 17:17 |
|
|
skay joined #sourcefu |
| 17:18 |
|
pdurbin |
sivoais: yeah, I'm talking about you: http://irclog.iq.harvard.edu/dvn/2014-02-08#i_6285 |
| 17:18 |
|
pdurbin |
and skay |
| 17:18 |
|
sivoais |
hehe |
| 17:18 |
|
pdurbin |
and protocol buffers |
| 17:18 |
|
skay |
pdurbin: I wasn't sure how much I'd actually chat, but didn't want to spend a lot of time offtopic in dvn |
| 17:18 |
|
skay |
so, my history (and orbitz's) with protocol buffers. |
| 17:18 |
|
pdurbin |
skay: how much you'd actually chat? ;) |
| 17:19 |
|
pdurbin |
semiosis is a good egg. likes this stuff |
| 17:19 |
|
pdurbin |
westmaas_: you only pay attention when I mention you |
| 17:20 |
|
pdurbin |
I never know when codex is listening :) |
| 17:20 |
|
pdurbin |
don't know who onder` is to be honest |
| 17:20 |
|
skay |
I only joined the company after about 5 years of their existence, so I wasn't involved withe original code base ... anyway, at the time we used jini for our transport layer (I think it is called Apache River now) and plain ol java serialization |
| 17:21 |
|
pdurbin |
javaeebot: lucky apache river |
| 17:21 |
|
javaeebot |
pdurbin: http://river.apache.org/ |
| 17:22 |
|
pdurbin |
"Apache River, a project furthering the development and advancement of Jini technology" |
| 17:22 |
|
skay |
they had a not bad service oriented archiecture (I'm not sure when that concept become called by that) |
| 17:22 |
|
pdurbin |
"Jini is a service oriented architecture that defines a programming model which both exploits and extends Java technology to enable the construction of secure, distributed systems consisting of federations of services and clients. Jini technology can be used to build adaptive network systems that are scalable, evolvable and flexible as typically required in dynamic computing environments." |
| 17:22 |
|
pdurbin |
skay: ok, please go on |
| 17:22 |
|
pdurbin |
:) |
| 17:23 |
|
skay |
nothing wrong with soa -- that's nice. jini is kind of obscure though. which isn't necessarily horrible but does mean anytie someone joins they have something new to pick up |
| 17:23 |
|
skay |
versus http, for example |
| 17:24 |
|
skay |
anyway, I can't remember exactly how service discovery/registry happens in jini |
| 17:24 |
|
|
ironcamel joined #sourcefu |
| 17:25 |
|
skay |
but vaguely, you can do it a different ways, I can't remember them all, some involve som local in memory caching or maybe you'd do it another way -- you can call out to the jini whatsit whose name I can't remember |
| 17:25 |
|
skay |
scaling problems with those, which I cannot all remember -- when something was stressed you'd get registration storms |
| 17:25 |
|
skay |
any time of failure could be tricky to troubleshoot |
| 17:26 |
|
skay |
I cannot remember all the obscure ways in which thigns would fail |
| 17:26 |
|
skay |
there was once some insane interacton with jini and some hardware |
| 17:27 |
|
pdurbin |
sure |
| 17:27 |
|
skay |
at some point (perhaps around the time I joined or a year after) people were feeling the pain and decided to swap out our transport/serialization mechanism (and also service registry) |
| 17:28 |
|
skay |
oh wait, another really horrible pain point -- hypothetically you'd want to be able to deploy services independently, but due to how people had been working, if some service revved,invariably the rest of the stack (or a lare part of it) would be required to rev |
| 17:28 |
|
skay |
head smack, right? |
| 17:28 |
|
skay |
might as well be a big ball of mude |
| 17:28 |
|
skay |
mud |
| 17:29 |
|
skay |
I don't think they should have blamed that on jini and pojos (shrug) but sure, I'll say maybe it was a contributor |
| 17:30 |
|
skay |
and so, we settled on an reverse proxy dispatcher thingee written in erlang, which we called, ha ha, erlgrey and the teakettle |
| 17:30 |
|
skay |
(get it? java? yeah) |
| 17:30 |
|
skay |
and the transport framework we called lapsang (lapsang souchang, right?) |
| 17:30 |
|
skay |
(my coworker suggested teabag but no one liked that) |
| 17:31 |
|
skay |
and then everyone went on a mission to slowly get all the teams to expose http services rather than jini services |
| 17:31 |
|
skay |
and with protocol buffers rather than pojos (or json) |
| 17:31 |
|
skay |
I'm not sure why the guys settled on protos rather than json for everything |
| 17:32 |
|
skay |
and I think some folks stilled use json. |
| 17:32 |
|
skay |
I spent a lot of time in the hotel stack (and on the team that handled the teakettle and other framework things) and by god you would nto want to use something json for some of those payloads |
| 17:32 |
|
skay |
using protos and lzh compression helped a lot |
| 17:33 |
|
skay |
plus people making some more lightweight apis so that they didn't send so much information over the wire |
| 17:33 |
|
skay |
I got a team where we were helping other teams convert their services to use the "lapsang framework" |
| 17:34 |
|
skay |
and then also ended up helping people with performance where we spent time looking where the bottlenecks where with the hotel stack |
| 17:34 |
|
skay |
and hence saw how big some of their payloads got -- worse case once was a market search for DFW, around 9 mb! |
| 17:35 |
|
skay |
haha pretty small, but not small for those messages |
| 17:35 |
|
skay |
showed a considerable impact on search results! |
| 17:35 |
|
skay |
we wanted to shave off 200 ms |
| 17:36 |
|
skay |
anyway, it looks like when deciding which serialization method to choose it can be risky because you don't want to settle on what might be a fad |
| 17:37 |
|
skay |
I don't now if protocol buffers is a fad. I'd definitely want to survey the landscape if I were you |
| 17:37 |
|
skay |
but right now for you, this is all hypothetically. I don't know what you need and json is probably great |
| 17:37 |
|
skay |
and you aren't an ecommerce site |
| 17:38 |
|
pdurbin |
erlgrey. nice. (catching up) |
| 17:38 |
|
skay |
that is all most of the chatting about this |
| 17:38 |
|
skay |
I didn't think I'd chat so much |
| 17:38 |
|
skay |
this might be the equiv of a table top roleplayer going over their character stats with you |
| 17:39 |
|
pdurbin |
:) |
| 17:40 |
|
pdurbin |
well, I was saying the mentioned protocol buffers at http://programmingthrowdown.blogspot.com/2012_04_01_archive.html but both those guys work (worked?) at Google |
| 17:40 |
|
skay |
erlang is fun by the way. I don't think they necessarily needed to do what they did with it |
| 17:40 |
|
pdurbin |
seems like companies come up with their own IDLs: http://en.wikipedia.org/wiki/Interface_description_language |
| 17:40 |
|
pdurbin |
Thrift by Facebook, for example |
| 17:41 |
|
pdurbin |
skay: anyway, keep talking and I'll catch up. need shower. need taking my 4 year old to a birthday party |
| 17:41 |
|
skay |
we tried to shift away from rpc like calls towards restful things where possible |
| 17:41 |
|
skay |
oh cool! weeee |
| 17:41 |
|
skay |
birthday |
| 17:42 |
|
skay |
do you have many fu channels? |
| 17:56 |
|
pdurbin |
skay: not so many: http://wiki.greptilian.com/haunts |
| 17:56 |
|
pdurbin |
dress like a princess ballet party |
| 19:32 |
|
codex |
pdurbin: i am always listening :) |
| 21:20 |
|
pdurbin |
codex: heh |
| 21:21 |
|
pdurbin |
party was fun: Phil Durbin - Google+ - my badass princess - https://plus.google.com/+PhilDurbin/posts/d19zkH8iVag |
| 23:14 |
|
westmaas_ |
pdurbin: haha, its true :( |
