| *** tpb has joined #melange | 00:00 | |
| *** SRabbelier1 has joined #melange | 00:31 | |
| *** SRabbelier has quit IRC | 00:31 | |
| *** SRabbelier1 is now known as SRabbelier | 01:01 | |
| *** ChanServ sets mode: +o SRabbelier | 01:02 | |
| *** madrazr has joined #melange | 01:57 | |
| *** ChanServ sets mode: +v madrazr | 01:57 | |
| *** madrazr has quit IRC | 03:58 | |
| *** madrazr has joined #melange | 04:43 | |
| *** ChanServ sets mode: +v madrazr | 04:43 | |
| *** madrazr has quit IRC | 05:55 | |
| *** madrazr1 has joined #melange | 05:55 | |
| *** madrazr1 is now known as madrazr | 05:55 | |
| *** ChanServ sets mode: +v madrazr | 05:56 | |
| *** scorche has quit IRC | 07:02 | |
| *** scorche has joined #melange | 07:24 | |
| *** madrazr1 has joined #melange | 08:51 | |
| *** madrazr has quit IRC | 08:51 | |
| *** madrazr1 has quit IRC | 09:01 | |
| *** madrazr has joined #melange | 09:01 | |
| *** ChanServ sets mode: +v madrazr | 09:01 | |
| *** Lennie has joined #melange | 09:19 | |
| *** ChanServ sets mode: +o Lennie | 09:19 | |
| *** Lennie is now known as Lennie|Gone | 09:24 | |
| danderson | SRabbelier: ping? | 10:16 |
|---|---|---|
| Lennie|Gone | danderson, not sure if he's awake yet since he's in the states :) | 11:18 |
| danderson | aaw | 11:18 |
| SRabbelier | danderson: pong | 12:05 |
| SRabbelier | danderson: export yes? | 12:05 |
| danderson | yes, if possible :) | 12:05 |
| SRabbelier | danderson: will do so now | 12:05 |
| danderson | I'm working on getting the 2005-2008 data out, but the 09 one is the most important :) | 12:05 |
| danderson | (reminder: successful students + all accepted mentors plz) | 12:06 |
| danderson | and I owe you a beer :P | 12:06 |
| SRabbelier | danderson: oh? | 12:06 |
| danderson | oh, and just their google account (email) is all I need | 12:06 |
| danderson | well, for helping me out on this :) | 12:06 |
| SRabbelier | danderson: ah, heh, I'd figure it'd be the other way around, :P | 12:06 |
| danderson | gotta run now, laterz | 12:07 |
| *** Lennie|Gone is now known as Lennie | 13:45 | |
| SRabbelier | danderson: mailed | 13:52 |
| danderson | SRabbelier: just got back, you win, beer is owed. | 13:54 |
| Lennie | danderson, I think SRabbelier would be happier with food or a cherry coke :P | 13:58 |
| danderson | whatever, that works too | 13:58 |
| SRabbelier | danderson: or some buttershots with baileys :P | 14:04 |
| SRabbelier | Lennie: how far did you test the conversion? | 14:04 |
| Lennie | all | 14:10 |
| Lennie | SRabbelier: baileys? really :) | 14:10 |
| Lennie | what made you change your midn :P | 14:10 |
| SRabbelier | Lennie: buttershots + baileys + grand marnier = a shot I drank with Jack, it's tasteh :P | 14:11 |
| Lennie | ok )O | 14:11 |
| Lennie | just don't feed SRabbelier beer | 14:11 |
| Lennie | that's all then :) | 14:11 |
| Lennie | oh and keep all liqour away from Erant :p | 14:12 |
| Erant | Don't you dare. | 14:13 |
| SRabbelier | Lennie: good idea, or he'll puke all over us again | 14:13 |
| Erant | So, who epic failed on the security. | 14:14 |
| Lennie | do i care :)? | 14:15 |
| Lennie | oh damn I do :P | 14:15 |
| Erant | Calls himself a project manager. | 14:16 |
| Erant | Speaking of managers, LH still in Vegas? | 14:16 |
| Lennie | no clue :P | 14:16 |
| *** Lennie has quit IRC | 14:17 | |
| Erant | Right. Bye? | 14:17 |
| *** madrazr has quit IRC | 17:09 | |
| danderson | SRabbelier: ping? | 18:05 |
| SRabbelier | danderson: pong | 18:06 |
| danderson | SRabbelier: what does the mentors list you sent include? | 18:06 |
| SRabbelier | danderson: all active mentors for this year | 18:06 |
| danderson | getting lots of 2009 mentors saying I misclassified them as invalid | 18:07 |
| danderson | oh gah | 18:07 |
| danderson | the list has [email protected] | 18:07 |
| danderson | but the account login is [email protected] | 18:07 |
| SRabbelier | danderson: tell them to use the same account that they use on Melange :P | 18:08 |
| danderson | I think it is the same | 18:08 |
| danderson | but that is the primary email registered for the account or something | 18:08 |
| danderson | anyway, we'll see how bad it gets | 18:08 |
| * SRabbelier nods | 18:08 | |
| danderson | I was just wondering if there was something obvious I misunderstood about your data dump :) | 18:09 |
| *** MatthewWilkes has joined #melange | 18:09 | |
| * SRabbelier shakes | 18:09 | |
| SRabbelier | danderson: no i think you got it right | 18:09 |
| danderson | ? | 18:14 |
| MatthewWilkes | danderson: Cheers :) (didn't want to email and clog up your inbox even more) | 18:22 |
| danderson | SRabbelier: does the admin data dump also list co-admins? (or backup admins or whatever it is they're called) | 18:36 |
| danderson | phew, finished beating back the first wave (pun not intended but kept after noticing :P) of "ohnoes i haz no wave invite" email | 19:05 |
| danderson | worse than a zergling rush | 19:05 |
| * danderson builds more bunkers | 19:06 | |
| MatthewWilkes | We'll get in either way :) | 19:06 |
| danderson | I'm already at 2% of my daily app push quota | 19:07 |
| danderson | just pushing fixes and updates :P | 19:07 |
| MatthewWilkes | danderson: Well, we're all very thankful! :) | 19:10 |
| danderson | okay and wtf is happening to my internet connection | 19:13 |
| danderson | it's laggy as shit all of a sudden | 19:13 |
| danderson | I think I'm going to have to blacklist my flatmate's MAC address again... | 19:13 |
| *** mithro has quit IRC | 19:23 | |
| SRabbelier | danderson: push to where quota? | 19:34 |
| danderson | to app engine | 19:34 |
| danderson | you have a quota of 1000 app pushes/day | 19:35 |
| danderson | I'm currently at 3% of that quota :P | 19:35 |
| danderson | (it's just simple DoS protection, nothing fancy) | 19:35 |
| MatthewWilkes | Awesome, "We have verified that you participated in SoC" | 19:35 |
| * danderson says "Access granted" in a very neutral robotic tone | 19:37 | |
| SRabbelier | danderson: what are you pushing? | 19:37 |
| danderson | updates to http://gsoc-wave-invites.appspot.com/ | 19:38 |
| tpb | Title: GSoC wave invite checker (at gsoc-wave-invites.appspot.com) | 19:38 |
| danderson | the valid/invalid lists are just two .txt files bundled in the app | 19:38 |
| danderson | and the app python module just reads them into a set on startup :P | 19:38 |
| danderson | epic cheap database! | 19:38 |
| danderson | but it means I have to repush when I tweak the lists, or fix bugs | 19:38 |
| MatthewWilkes | danderson: You should put ads on the sign-in page and leak the address | 19:38 |
| danderson | haha | 19:39 |
| danderson | post it to reddit | 19:39 |
| MatthewWilkes | now, come on | 19:39 |
| SRabbelier | danderson: lol, cute, you have no life :P | 19:39 |
| danderson | and exceed my daily app engine quota in 5 minutes | 19:39 |
| MatthewWilkes | reddit is only used for bitching about python infighting | 19:39 |
| MatthewWilkes | (anyone on python-dev or distutils-sig will know what I mean) | 19:39 |
| SRabbelier | danderson: why not use the database though? :P | 19:40 |
| danderson | because it's 1000 entries, ridiculously small | 19:41 |
| danderson | because I need a .txt with one email per line to send to the wave team | 19:41 |
| danderson | and because I didn't want to waste time fucking around with the datastore API | 19:42 |
| danderson | my "database access layer" is the following line of python: | 19:42 |
| SRabbelier | danderson: create one entity per list? | 19:42 |
| danderson | VALID = set(l.strip() for l in open('data/valid.txt')) | 19:42 |
| SRabbelier | danderson: lol :) | 19:43 |
| danderson | my "data access method" to verify an account is: | 19:43 |
| danderson | if user.email().lower() in VALID: | 19:43 |
| danderson | try that with the datastore | 19:43 |
| danderson | plus, with this, I can respond to queries in-memory | 19:43 |
| MatthewWilkes | class DataStoreWrapper(object): def __contains__ … | 19:43 |
| danderson | no external round-trips, just a lookup in a local hashtable, and boom | 19:43 |
| danderson | plus, RAM is free | 19:43 |
| danderson | datastore lookups are slow, prone to long-tail failures, and are charged against quota | 19:44 |
| SRabbelier | danderson: replace open() with db.get('valid').data | 19:44 |
| SRabbelier | danderson: cache the result | 19:44 |
| danderson | yes, but you're missing the critical point: why? | 19:44 |
| danderson | I have a .txt that fits in memory | 19:44 |
| SRabbelier | danderson: because you can? | 19:44 |
| MatthewWilkes | SRabbelier: I'm with danderson on this, flat text files are my 2nd favourite database format | 19:44 |
| danderson | I don't need to scale, I'll be deleting the app in 2 days | 19:44 |
| danderson | the datastore is awesome at scaling | 19:44 |
| SRabbelier | danderson: I guess this does deserve some 'awesomeness' credits :P | 19:45 |
| danderson | but to present an interface to a 1k element hashtable? Seriously? | 19:45 |
| danderson | memory wins. | 19:45 |
| danderson | I could also cite our internal document reminding engineers of orders of magnitude in computing | 19:45 |
| SRabbelier | danderson: oooh? | 19:45 |
| SRabbelier | 0, 1, many? | 19:45 |
| danderson | well, except I'm not allowed to, obviously | 19:46 |
| danderson | but I can give you one extract | 19:46 |
| * MatthewWilkes wishes there was a site to sign up for the chance to hug Lars Rasmussen… | 19:46 | |
| danderson | main memory reference: ~100ns | 19:46 |
| *** mithro has joined #melange | 19:46 | |
| *** ChanServ sets mode: +v mithro | 19:46 | |
| danderson | round trip within local network: ~500 000ns | 19:46 |
| danderson | just saying :P | 19:46 |
| SRabbelier | danderson: point, set, match :P | 19:47 |
| danderson | the most awesome part of that is that disk space is about 3x slower than reading the data from RAM on another machine :) | 19:48 |
| SRabbelier | danderson: oh, how slow is disk access then? | 19:48 |
| danderson | so, if you have the choice between a cache on disk, or an in-memory cache on a LAN machine, take the LAN machine, every time | 19:48 |
| danderson | it's actually around 100x slower because it has high setup costs | 19:49 |
| danderson | it's pretty good at streaming lots of data once it's wasted a long time setting up | 19:49 |
| danderson | the problem is you don't usually want to read more than <1M at a time | 19:49 |
| danderson | in which case the setup cost of moving the heads, waiting for the data to arrive, etc. are actually really significative | 19:49 |
| SRabbelier | danderson: in your case most certainly so :P | 19:49 |
| danderson | well, in my case yes, but also in most cases | 19:50 |
| danderson | there are two forms of data access in general | 19:50 |
| danderson | sequentially stream lots of data (eg. transfer 10G of video from disk to disk) | 19:50 |
| danderson | or randomly stream small pieces of data | 19:50 |
| danderson | disks are not too bad at the former | 19:50 |
| danderson | but suck rocks at the latter | 19:50 |
| danderson | because the setup cost takes longer than actually reading the data | 19:51 |
| SRabbelier | danderson: even with SSD? | 19:51 |
| danderson | things to keep in mind while designing a distributed system ;) | 19:51 |
| danderson | with SSD, not sure what the setup cost is exactly, but it does change the game quite a lot | 19:51 |
| danderson | it'll still be slower than ram access, but dunno about remote ram access | 19:52 |
| SRabbelier | danderson: interesting | 19:52 |
| danderson | but then you can just change the problem into randomly writing small pieces of data | 19:52 |
| danderson | and end up in a world of shit again :) | 19:52 |
| danderson | SSDs are catching up, but you can't beat ram's bandwidth atm | 19:52 |
| SRabbelier | danderson: uhm | 19:52 |
| SRabbelier | danderson: I thought the newer SSD's don't have that problem | 19:52 |
| SRabbelier | danderson: http://www.codinghorror.com/blog/archives/001304.html | 19:52 |
| tpb | <http://ln-s.net/4PtK> (at www.codinghorror.com) | 19:52 |
| danderson | they don't have that problem when you consider them to be disks, yes | 19:53 |
| danderson | so for disks, they have decent read/write performance | 19:53 |
| danderson | but compared to ram, I'm pretty sure the write performance still sucks :) | 19:53 |
| SRabbelier | danderson: ah, but compared to RAM or even network still sucky | 19:53 |
| * SRabbelier nods | 19:53 | |
| danderson | well, maybe not network, that's the thing | 19:53 |
| danderson | they may be good enough to become an alternative to the network | 19:54 |
| danderson | but then you have different problems | 19:54 |
| danderson | what sizes are SSDs these days? | 19:54 |
| danderson | I mean, what's the cost per MB? | 19:54 |
| danderson | ie. how much is it going to cost you to scale up? | 19:55 |
| SRabbelier | 128GB costs around $360 | 19:56 |
| danderson | ok, not too bad, still expensive compared to ram though | 19:58 |
| danderson | it'd be interesting to update our numbers to include read from local SSD and read from remote SSD | 19:58 |
| SRabbelier | indeed | 19:58 |
| SRabbelier | and I reckon those prices get lower if you buy in bulk :P | 19:58 |
| danderson | because I don't know how much of the disk cost is also caused by the slowness of the data bus | 19:58 |
| danderson | compared to the FSB for ram | 19:58 |
| * SRabbelier nods | 19:58 | |
| danderson | ok, seriously, wtf | 20:21 |
| danderson | it looks like my ISP is dropping all UDP traffic or something | 20:21 |
| danderson | zero DNS resolution | 20:21 |
| SRabbelier | danderson: nice, try OpenDNS instead? | 20:55 |
| *** tansell_laptop has joined #melange | 22:12 | |
| durin42 | danderson: this is the internet telling you to go to sleep ;) | 23:40 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!