greptilian logo

IRC log for #openknot, 2015-07-24

Open Knot Communications Platform - https://github.com/openknot

| Channels | #openknot index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
00:31 pdurbin dotplus: I don't understand how https://zulip.com/api/endpoints/ has anything to do with "threaded IRC"
02:06 prologic btw guys
02:06 prologic been thinking more about this
02:06 prologic I have a few things I'd like to finish up first
02:06 prologic but I *believe* we should get cracking on a prototype for openknot
02:06 prologic as quickly as possible
02:12 dotplus do you have an opinion about what should be prototyped first?
02:27 prologic the whole system I believe
02:27 prologic as a minimal functional system
02:27 prologic So I think this means
02:27 prologic let's build the backend and client/frontend
02:28 prologic get emails into it, get irc into it and web-style forum
02:28 prologic even if the client/frontend looks rubbish that's okay; but let's build a prototype!
02:31 dotplus how about email/irc and a web archive? in other words, email & IRC interfaces are bi-directional, web is presentation only to start with. that should make it a little easier
02:32 dotplus are you envisaging a single daemon that listens for SMTP on TCP25, IRC on TCP6667 & HTTP(S?) on TCP80/443?
02:33 dotplus (just talking about the prototype for now)
02:37 dotplus do you think we should write from scratch? presumably with an existing framework in the language of choice (such as circuits:) or build on top of existing transport such as AMQP (rabbitmq?) or XMPP (prosody?)
02:42 prologic yeah agreed
02:42 prologic let's do that
02:42 prologic we can then add the bidireciton layer inot the web clienet later
02:43 prologic re frameworks/libraries/protocols
02:43 prologic that's a very good question
02:43 prologic the new UNIX Philosopher in me says we *should* reuse as much software and code as we can leverage
02:44 dotplus new? you recently came to this? I think not:)
02:44 prologic however as I feel this is a kind of "Third System" for solving what has become an apparent broken and disparate communications medium
02:44 prologic I think we *may* have come up with some new ideas
02:44 prologic no not new lol
02:45 prologic new as-in; finally reading the damn book!
02:45 prologic haha
02:45 prologic I *have* always believed in many of the key UNIX tenets
02:45 prologic I think if we can break down the first prototype
02:46 prologic and do it the UNIX way :P
02:46 prologic we can make this an attractive idea and First System and well on our way to the 2nd/3rd systems (quickly) :P
02:47 dotplus which book? TAOUP? Advanced Programming in the unix environment? something else?
02:50 prologic nah
02:50 prologic UNIX and the Linux Psephology
02:51 prologic by the same guy that wrote the original: UNIX Philosophy
02:55 dotplus not familiar with that one.
02:57 dotplus I'll have to look for it. Anyway, I'm not sure where you're leaning with respect to: "from scratch" vs rabbit/prosody. there are advantages and disadvantages in both directions in my mind.
02:57 prologic oh? :)
02:57 prologic but anyway I have the one you mentioned on my reading list too
03:00 dotplus for me, the art of unix programming by esr was a read-it-linearly, the Stevens book is more of read/skim a/some chapters, refer to it when required. But I'm not a real programmer, let alone a real systems programmer.
03:04 dotplus I'm tempted by AMQP/rabbitmq. a) it's already written b) really solid c) well maintained d) performant e) we can write individual messengers in more or less any language benefitting from existing libraries
03:13 prologic are we think of a distributed system here
03:13 prologic hence RabbitMQ,etc?
03:14 dotplus yes. I think this needs to be a distributed system from the beginning.
03:21 prologic RabbitMQ it is then
03:21 prologic so we'll be using that as our protocol as such
03:21 prologic I thinking Redis as our data store
03:22 dotplus because we're serializing everything to json?
03:22 prologic and if we want to leverage an existing concurrency framework I have written before (and can probably do so again easily; a circuits+rabbitmq adapter; for some reason I lost that :P) -- and integrating Redis is trivial (but it would also be nice to integrate with it's pub/sub)
03:23 prologic I think we should
03:23 prologic JSON is a pretty portable data format
03:23 prologic I agree RabiitMQ will give us a big advantage in the distributed messageing
03:23 prologic and Redis will give us a big advantage in high performance storage
03:24 dotplus redis also seems to satisfy the criteria in a-e above:)
03:24 prologic a-e?
03:24 dotplus "a) it's already written b) really solid c) well maintained d) performant e) we can write individual messengers in more or less any language benefitting from existing libraries"
03:25 dotplus I'm guessing from reputation that redis satisfies those although I have little experience with it
03:32 dotplus ok, we've made some progress, awesome. But I need to get ready to sleep. I'll commit a brief write up of these decisions, some of the reasoning behind them, perhaps some docs on how to get going and/or ansible playbooks (roles?) for rabbitmq/redis over the next few days.
03:34 dotplus bear: I know you have an XMPP/prosody axe to grind. If you think we're making a big mistake here, now would be a really good time to chime in with why and how to avoid it:)
03:46 prologic sounds good
03:46 prologic -1 on the ansible playbooks though
03:46 prologic I was thinking more Docker
03:46 prologic dev -> prod with continuous delivery (as opposed to contentious deployment)
09:57 sivoais joined #openknot
10:07 sivoais joined #openknot
11:39 dotplus if you like, I don't really use docker yet, I probably wouldn't do that as well/as quickly. but it's probably about time I had an excuse to learn.
11:41 prologic :)
11:42 prologic if not Docker; at least the Docker way
11:42 prologic i.e: build images that folks can just spin up
11:54 pdurbin prologic: war files ;)
12:28 prologic lol
12:30 pdurbin https://en.wikipedia.org/wiki/Web_container
12:34 dotplus pdurbin: I know you're a Java person so I can't tell whether you're serious or not
12:36 pdurbin heh. what's interesting is that in the java world the container is tomcat but in the docker world the container is the app
12:37 pdurbin right?
12:37 pdurbin in java the app is a war file and you deploy it to a container such as tomcat
12:40 pdurbin "A container is a runtime instance of a docker image" http://docs.docker.com/reference/glossary/#container
12:40 prologic pdurbin, correct
12:40 prologic it's not just "web"
12:40 pdurbin so in java and docker "container" means very different things
12:40 prologic in the Java world things are almost hard tied to Tomcat almost
12:40 prologic :)
12:40 prologic yes
12:41 prologic a container here is strictly speaking an isolated process
12:41 prologic or group of processes
12:41 prologic with their own cpu, memory and network resources
12:42 dotplus there is that confusion/disconnect about the use of the word container. But also rememer docker is really just lxc: https://en.wikipedia.org/wiki/LXC. so the app is the (collection of) services running inside the container.
12:43 prologic well it used to be lxc - yes  :)
12:43 prologic now it talks directly to the kernel
12:43 prologic but essentially cgroups
12:43 prologic the bigger picture for me has always been that you can "package" up an app/service/whatever
12:43 prologic and have it work everywhere
12:44 prologic no matter what it is really
12:44 prologic no dependency hell or having to have certain system libraries
12:44 dotplus conceptually it's still the same as lxc/cgroups even if they rewrote the implementation to not actually use lxc
12:44 prologic no version conflicts between various things on your system
12:44 prologic repeatability
12:44 prologic yeah yeah
12:44 dotplus right
12:44 prologic I think folks will forget about lxc eventually though :)
12:45 prologic cgroups ftw
12:45 dotplus perhaps. perhaps there will be new hotness that replaces docker.
12:45 prologic pdurbin, think of it as more of the equivilent of Java JAR
12:45 prologic but on a broader scale :)
12:45 prologic JAR(s) are more aligned with Ruby Gems, Python Eggs/Wheels, etc
12:46 prologic but kind helps to think about it that way?
12:46 prologic dotplus, there already is :)
12:46 prologic runC
12:46 dotplus we kinda spiralled offtopic here
12:46 pdurbin I think I get how Docker works. I'm just trolling. :)
12:46 prologic and the OCF
12:46 prologic haha
12:46 prologic yeah it's all pdurbin's fault :)
12:49 dotplus ah yes, I heard briefly about runc/ocp. Is that a direct competitor to docker (even if it's not as mature/featureful yet)?
12:49 prologic no not at all
12:49 prologic in fact Docker contributed their container runtime directly to the project
12:49 prologic as runC
12:50 prologic I believe everyone wants to build an open standard for containers
12:50 prologic such that a container/image built with Docker can run on any runC compatible platform
12:50 prologic that's my take on it
16:29 bear rabbitmq will be fast at distributing yes, but it won't do intelligent routing based on presence which is at the core of XMPP
16:33 dotplus right. but are we interested in presence notification? any node subscribes to what it wants (when it wants)
16:49 bear presence allows the xmpp server to know if it's sending the message (your online) or a notification you have a message (your offline)
16:49 bear the difference is if the distribution is to people or to software
16:52 bear the need for a raw backend message pump (rabbitMQ or Redis) doesn't remove the usefulness of Prosody (XMPP) for client delivery
17:57 dotplus ah. that makes sense. rabbitmq would be the raw message broker, redis would be the message store for local, prosody would be between rabbitmq & xmpp clients and in the same way an irc daemon would be between rabbit & irc clients.
17:57 dotplus this is getting clearer
19:33 bear I would also not use rabbitmq in the beginning, it has a more involved setup -- if all that is needed is a topic based pub/sub then redis can do that
22:19 prologic I actually kind of agree with bear here
22:19 prologic If we just use Redis say + circuits
22:19 prologic we get both anyway
22:19 prologic circuits has distributed processing facilities
22:19 prologic and if we want to swap that out or integrate it with RabbitMQ/ZeroMQ later on we can
22:34 prologic maybe I'll mock/prototype up somethig tonight?
22:36 pdurbin what would the prototype do?
22:37 prologic accept email delivery
22:37 prologic have an irc server
22:37 prologic hmm I'll have to re-read the backlog of what dotplus and I decided
22:37 prologic I think we said we'd leave the web ui/forum out for the time bging
22:46 dotplus it would be nice to have a web interface to stored messages, if not for sending them yet
22:46 dotplus but not a forum, just simple presentation of the archive
22:46 bear stored message web view is a lot (IMO) like a irc log view
22:47 dotplus yep
22:47 bear irc, xmpp, email -> router
22:47 bear and then have an archiver listening to the router
22:47 dotplus right
22:49 dotplus that was what I was envisaging last night when I was talking about rabbit. but if it's easy to the router in circuits/redis AND easy to swap out later, that's fine too.
22:55 pdurbin so wait. I send an email and a message shows up in an IRC channel? with a link to the archived message?
23:05 bear it could
23:06 bear the pop3 or imap part would notice the event and broadcast it
23:06 bear the archive receives everything and stores it
23:06 bear because everything would have a UUID adding associations to it is a matter of broadcasting an action to the UUID
23:14 pdurbin so it'd be configurable
23:18 bear in my mental model the archiver stores *everything*
23:19 bear it's like a transaction log
23:20 bear {"event": "create", "id": "UUID_UUID_####", "item": { details of email }, "source": "something to tell it came from the email agent"}
23:22 pdurbin everything is an event
23:23 bear for this part of the backend yea
23:26 pdurbin and would I be able to reply to that email via IRC?
23:28 bear that is where the fun starts
23:29 bear your email agent would need to listen events
23:29 bear and then get irc to be able to send the event
23:30 bear technically a reply is a message to an ID that represents the email
23:30 bear so the email agent would check the ID, discover it's an email and craft a reply
23:30 bear a new email would be a post against a contact with (waving hands here) a marker that said you wanted to use their email address as the target
23:50 pdurbin huh. I didn't think I'd be driving my email client via IRC.
23:51 pdurbin I thought maybe I'd talk to a bot in an IRC channel who would send a message to the email thread. Or something. :)
23:53 bear yea, to be blunt I haven't implemented that idea yet - it was just something I've always thought of
23:54 bear to keep the OO nature of things going - fire events off of items to get things done
23:55 pdurbin but the idea is still that it's one thread or conversation or whatever, right? whether people are discussion via email or IRC
23:57 bear yes
23:57 pdurbin I think some user stories might help.
23:57 bear agree with that completely

| Channels | #openknot index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Open Knot Communications Platform - https://github.com/openknot