So I have some downtime in the hospital with my son and don't really have much motivation to do anything at the moment...so, why not blog?
I've been hired recently by Vanderbilt University's Center for Human Genetic Research in which I'll be responsible for some super sweet SNP (single nucleotide polymorphisms or "snips") searching mechanisms and I will also be responsible for creating a service for sharing this information with other universities in what I can only think of as a "genetic research alliance" of sorts.
I am really looking forward to see what I learn and can apply to this job, hopefully I can put something together that helps the researchers here find something useful for humanity or even just my son!
What I've found amazing thus far is how much of an educational investment they (Vanderbilt) are willing to pump into you if you are up to it. They allow for "auditing" of classes which pertain to your field of work, so for me (from what I understand) I'll be auditing a biological statistics class and a genetics class come January (or whenever the next semester starts, but I think that's right).
This whole realm of genetic information has really put my mind at work in terms of storage (we are talking petabytes here) and retrieval. Something that's crossed my mind is the thought of consistent hashes almost like bookmarks in an individual's sequence. I'm sure there are flaws in thinking of it this way, but hey, it's a start. And if nothing else, I'll learn more from the failures anyways.
Which leads me to my next exciting topic. Since it is research work, there is of course the possibility of trying out some of the non-standard (read: "new fangled and non-relational") storage systems in order to see if they function better for searches over current methods. Sounds like a real drag, eh?
Hopefully I can regurgitate more than less in regard to the specifics of what I'm working on in the coming years. The amount of data and the structures within it are truly a rather amazing challenge the more I think about them.
Time for the rambling to end, it's almost time for T's late night bath - we're up to every 12 hours to wash the chemo off of him now, so that's one of the first steps towards getting him home.
Thursday, October 15, 2009
Wednesday, August 5, 2009
Benefit Reminder Aug 7th 5pm at Tin Roof Nashville
This Friday, August 7th from 5pm to 9pm at the Tin Roof in downtown Nashville, TN we will be holding my son's benefit concert.
Details are below, clipped from the electronic flier our wonderful organizer put together:
***CONFIRMED ARTIST FOR AUGUST 7TH!!!!!***
Richard Marx has so graciously offered to fly in for our event and sing!!! We are thrilled and can't wait to hear his awesome music!!! Thank you Richard!!!!
He has also invited two special guests that you just might know...Danielle Peck and Coley McCabe!!! Can't wait to hear them all on Friday!!!
***Confirmed Independent Artists***
Michael David
Erika Jo
Kelsie and Ryan
Joe Wiese
Clayton Quisenberry
Sarah Lenore
Brian Desveaux
Shanna Crooks
Gary Slayton
Holly Bakehorn
Devin Belle
Candice Crain
There will also be a silent auction full of awesome items such as gift certificates to local restaurants, clothing stores, Titans and Predators merchandise, autographed 8x10's, posters and t-shirts signed by the artists (big names!!) and so much more!
We would love to pack the Roof and raise a ton of money for this family...I can't even imagine what they are going through right now...please keep them in your thoughts and prayers. We hope to see you on the 7th!!!
*Even if you can't make it out to this event and would like to donate, please see the information below and you can give directly to the bank where there is a fund set up for my son.
Also if you want a support bracelet ($5) you can contact me at bradswinfrey over at yahoo dot com.
Details are below, clipped from the electronic flier our wonderful organizer put together:
***CONFIRMED ARTIST FOR AUGUST 7TH!!!!!***
Richard Marx has so graciously offered to fly in for our event and sing!!! We are thrilled and can't wait to hear his awesome music!!! Thank you Richard!!!!
He has also invited two special guests that you just might know...Danielle Peck and Coley McCabe!!! Can't wait to hear them all on Friday!!!
***Confirmed Independent Artists***
Michael David
Erika Jo
Kelsie and Ryan
Joe Wiese
Clayton Quisenberry
Sarah Lenore
Brian Desveaux
Shanna Crooks
Gary Slayton
Holly Bakehorn
Devin Belle
Candice Crain
There will also be a silent auction full of awesome items such as gift certificates to local restaurants, clothing stores, Titans and Predators merchandise, autographed 8x10's, posters and t-shirts signed by the artists (big names!!) and so much more!
We would love to pack the Roof and raise a ton of money for this family...I can't even imagine what they are going through right now...please keep them in your thoughts and prayers. We hope to see you on the 7th!!!
*Even if you can't make it out to this event and would like to donate, please see the information below and you can give directly to the bank where there is a fund set up for my son.
Also if you want a support bracelet ($5) you can contact me at bradswinfrey over at yahoo dot com.
Hardship
It's not often that I blog about life events (or at least it doesn't feel that way) but this one, or rather this series of events certainly requires some ventilation.
In May of '09, we were noticing that my (at the time) youngest son was walking a bit wobbly, not constantly however, so perhaps he was just growing into himself at the time as he had grown a great deal in height the last few months. I really wish that had been the case.
No. Instead, the only thing growing which, in turn was causing his wobbly gait was in fact a tumor. More specifically a cancerous tumor classified as Medulloblastoma.
I remember it very clearly, taking him in for his MRI and making a remark to my wife that we should take him out for a little present for the rough morning he had been through. Heh, well that was wishful thinking wasn't it? They brought Thomas in from his MRI (he was out from his anesthetic still) and began to wake him up a bit. About five minutes or so after he was awake again a team of two doctors came in to tell us that he had a tumor roughly about the size of a golf ball growing between his brain stem and his cerebellum. My mouth (from what my wife tells me) dropped. My wife, 8 months pregnant at the time bursts into tears. My son, the sweet little thing that he is tells his mother that "it's ok, what's wrong?". I calm her down while still talking to the doctors standing in our little 5x12" recovery room. They said that they specialize in these specific types of tumors and that they require immediate resection and typically treatment. But that certainly doesn't just make it all better or tolerable, it just means that you are putting your feet down on a path that leads you down a very, very long journey where you don't know what the end result is.
Shortly thereafter, we were taken up to the ER to wait while they prepared a room in the critical care unit at VCH. Numerous people were in and out of our room to explain what was about to go on, things we should be doing to help in the meantime...it was all quite overwhelming as one can imagine. We were then taken up to our new temporary home on the fifth floor where they monitored every little aspect of our little boy for the next 72 hours until his surgery.
I'd like to share this short story for a moment before I explain the surgery. The day prior to his surgery, my wife and everyone else who had come down to help out had left the confines of our room and floor to get some fresh air and try to re-energize for the long day ahead. I stayed back to be with my son for some quality time. We played with his x-men and star wars figures for a while, making the usual little jokes we have with one another. When all of the sudden he slowly stops playing, looks up at me with a look I couldn't even describe and says "Daddy, I'm scared..." in his then slightly delayed little voice. I immediately responded to him by saying "you have nothing to worry about buddy, you are going to get through this, because you're the toughest little guy I know". He smiled a little and flexed his little muscles like the Incredible Hulk. I excused myself from our little battle to walk over to the window where I began to feel control of every bit of sanity I had slip away. I did everything I could to hold back the overwhelming feelings of fear and uncertainty to keep him calm and to not show him that I was sad or fearful. I had to be strong for him; really for everyone in our family; even every one of our friends and their outstretching networks of people who had heard of what happened to my child.
I had to stop. HAD TO. I couldn't. A few short outbursts of mumbled crying mixed with what I can only describe as hysteria leaped out from my chest and through my eyes. I held it in so hard that for the rest of the day I had the worst pressure headache throughout my entire skull. Between that, the lack of nourishment from stress and the lack of sleep I had to remind myself. This was nothing compared to the awful things going on inside my little boys head right now. So it passed...and I endured.
Back to it...
It was now 4:30am Monday morning, my son's surgery was scheduled for 8:00am. At 7:30 that morning they came and got him ready for his long day and explained to me the risks of the surgery which I was about to allow these strangers to perform on my sons brain. HIS BRAIN! Stroke, loss of facial movements (smiling, blinking, etc...), loss of ability to swallow, balance issues, death. Sounds like a decent trade off right? I signed the agreement. Once I saw them ship him off in his little crib, head marked for where incisions would be made (thankfully they only needed one on the back of his head) I remember just being completely numb. Everything tasted the same that day, nothing was funny, no one was consoling enough, sleep wasn't possible, nothing was possible until I could see my child alive after his surgery. Which eventually did come. We received an update every hour, always positive from one of the nurses. When finally at about 4:30pm that same day they told us they were done and that he would be in the recovery room soon.
His surgeon came into the waiting room where we were and brought us into the consultation room to brief us on what had gone down. So I'll relay that information here:
A single incision was made to the back of my sons head. They then removed a piece of his skull in a square fashion in order to have room to operate. Once they were inside, they lifted the folds of the cerebellum and could see the tumor. From what they saw, it had not gotten tangled in his brain stem or invaded any other space in his brain. They were actually able to remove the entire tumor without actually cutting ANY of his brain tissue. Thank God.
When we were finally able to see Thomas he laid in his little crib looking very sweaty and exhausted. His face was incredibly swollen (I believe he was upside down for his surgery) and a little raw by one of his cheeks from the shifting around in surgery. But for the most part he looked the same as he ever did with the exception of the throbbing incision on the back of his head.
For the next week we got nothing out of him, not head shakes, not ughs or grunts just short bursts of sheer frustration and screaming. He was trapped, knowing full well what he wanted to do (seemingly) but unable to perform it. Another week went by, he'd have his eyes open more often, I had him reach out and pick a spiderman figure from my hand one day, he wouldn't do that for another two weeks. The only word he ever really spoke in the office was to his older brother, who brought him a Bumble Bee Mr. Potato Head toy to which he said "thanks". That again, would be the only thing he would say for about the next two weeks.
After that 2-3 week span he would surprise us each day. He had gone from not being able to hold his own head up or even being able to swallow to swallowing and actually LAUGHING. This may sound cliche, but the day I was able to get him to laugh seemed to really trigger a whole new list of things that he was able to do again. He began drinking in very small amounts (he had a feeding tube this whole time) and a week later after ripping out his feeding tube, he ate. And he would not stop eating from there on out until his 15th day of chemotherapy.
So here we are almost two months into this, he's begun his second cycle of his first treatment of chemotherapy (3 cycles in 1 treatment - each cycle is 21 days long). His hair has thinned significantly and he will most likely be completely bald soon. He has already fought off one infection. Just another testament to this little boy's strength. He's most often in good spirits while we are in the hospital, it was a blessing having him home for more than 28 hours (unlike the day he got his infection and we had to take him right back).
We recently welcomed our third child into this world on the 24th of July. So now we don't only have the privilege of taking care of a soon to be 3yr. old and a 5 year old but a newborn to keep us uh..."company" all night long (just kidding the wee one has been great). It's been a new hurdle every few weeks, for instance, just today (Aug. 7th) my oldest son started kindergarten. It will make things easier during the day, but not so much when he needs to be somewhere safe and loving after school. We'll take care of it. We have to!
Wrapping up, I can say that since the time my wife broke down when they told us our sons diagnosis, (with the exception of the choked back fear and anger in the hospital room that one day) I have not had an adequate means of ventilation. I thought perhaps this would help, but it really hasn't. I have not yet told my wife the story about him saying he was scared, I'm afraid it would tear her up too much (she doesn't read this blog, it bores her). Sorry, just had another support person drop off dinner for us. That's another thing, the support network of people we've had has been simply outstanding. Mowing the lawn (it's a big, very steep yard), cooking dinners, doing laundry, shuttling our oldest son places and so on, it's been a great, great help.
If there's one thing I've learned in all of this it's that God won't give me more than I can handle. I am intrigued to see just how far that limit goes as I believe it has been pushed much further than I would have expected. Once we get through this, I feel that my family from our newborn on up will be stronger collectively for it. Especially my little trooper.
I'll follow up this post with information regarding the benefit concert a friend of ours (who just today found out her father was diagnosed with brain cancer) has put together here in Nashville for our son to raise some money for his medical expenses (don't worry, we are insured, but insurance does not cover 100% obviously).
If you read all of this, I applaud you. It's tough to hold onto a though and go with it so I bet this was probably a choppy read...
Thanks again to everyone who has helped out through this and even those of you who just give me that pat on the back when it's clear things are rough. I can't say again how lucky I am to have such a great network of friends, family and even just online acquaintances.
Thank you all.
Brad
In May of '09, we were noticing that my (at the time) youngest son was walking a bit wobbly, not constantly however, so perhaps he was just growing into himself at the time as he had grown a great deal in height the last few months. I really wish that had been the case.
No. Instead, the only thing growing which, in turn was causing his wobbly gait was in fact a tumor. More specifically a cancerous tumor classified as Medulloblastoma.
I remember it very clearly, taking him in for his MRI and making a remark to my wife that we should take him out for a little present for the rough morning he had been through. Heh, well that was wishful thinking wasn't it? They brought Thomas in from his MRI (he was out from his anesthetic still) and began to wake him up a bit. About five minutes or so after he was awake again a team of two doctors came in to tell us that he had a tumor roughly about the size of a golf ball growing between his brain stem and his cerebellum. My mouth (from what my wife tells me) dropped. My wife, 8 months pregnant at the time bursts into tears. My son, the sweet little thing that he is tells his mother that "it's ok, what's wrong?". I calm her down while still talking to the doctors standing in our little 5x12" recovery room. They said that they specialize in these specific types of tumors and that they require immediate resection and typically treatment. But that certainly doesn't just make it all better or tolerable, it just means that you are putting your feet down on a path that leads you down a very, very long journey where you don't know what the end result is.
Shortly thereafter, we were taken up to the ER to wait while they prepared a room in the critical care unit at VCH. Numerous people were in and out of our room to explain what was about to go on, things we should be doing to help in the meantime...it was all quite overwhelming as one can imagine. We were then taken up to our new temporary home on the fifth floor where they monitored every little aspect of our little boy for the next 72 hours until his surgery.
I'd like to share this short story for a moment before I explain the surgery. The day prior to his surgery, my wife and everyone else who had come down to help out had left the confines of our room and floor to get some fresh air and try to re-energize for the long day ahead. I stayed back to be with my son for some quality time. We played with his x-men and star wars figures for a while, making the usual little jokes we have with one another. When all of the sudden he slowly stops playing, looks up at me with a look I couldn't even describe and says "Daddy, I'm scared..." in his then slightly delayed little voice. I immediately responded to him by saying "you have nothing to worry about buddy, you are going to get through this, because you're the toughest little guy I know". He smiled a little and flexed his little muscles like the Incredible Hulk. I excused myself from our little battle to walk over to the window where I began to feel control of every bit of sanity I had slip away. I did everything I could to hold back the overwhelming feelings of fear and uncertainty to keep him calm and to not show him that I was sad or fearful. I had to be strong for him; really for everyone in our family; even every one of our friends and their outstretching networks of people who had heard of what happened to my child.
I had to stop. HAD TO. I couldn't. A few short outbursts of mumbled crying mixed with what I can only describe as hysteria leaped out from my chest and through my eyes. I held it in so hard that for the rest of the day I had the worst pressure headache throughout my entire skull. Between that, the lack of nourishment from stress and the lack of sleep I had to remind myself. This was nothing compared to the awful things going on inside my little boys head right now. So it passed...and I endured.
Back to it...
It was now 4:30am Monday morning, my son's surgery was scheduled for 8:00am. At 7:30 that morning they came and got him ready for his long day and explained to me the risks of the surgery which I was about to allow these strangers to perform on my sons brain. HIS BRAIN! Stroke, loss of facial movements (smiling, blinking, etc...), loss of ability to swallow, balance issues, death. Sounds like a decent trade off right? I signed the agreement. Once I saw them ship him off in his little crib, head marked for where incisions would be made (thankfully they only needed one on the back of his head) I remember just being completely numb. Everything tasted the same that day, nothing was funny, no one was consoling enough, sleep wasn't possible, nothing was possible until I could see my child alive after his surgery. Which eventually did come. We received an update every hour, always positive from one of the nurses. When finally at about 4:30pm that same day they told us they were done and that he would be in the recovery room soon.
His surgeon came into the waiting room where we were and brought us into the consultation room to brief us on what had gone down. So I'll relay that information here:
A single incision was made to the back of my sons head. They then removed a piece of his skull in a square fashion in order to have room to operate. Once they were inside, they lifted the folds of the cerebellum and could see the tumor. From what they saw, it had not gotten tangled in his brain stem or invaded any other space in his brain. They were actually able to remove the entire tumor without actually cutting ANY of his brain tissue. Thank God.
When we were finally able to see Thomas he laid in his little crib looking very sweaty and exhausted. His face was incredibly swollen (I believe he was upside down for his surgery) and a little raw by one of his cheeks from the shifting around in surgery. But for the most part he looked the same as he ever did with the exception of the throbbing incision on the back of his head.
For the next week we got nothing out of him, not head shakes, not ughs or grunts just short bursts of sheer frustration and screaming. He was trapped, knowing full well what he wanted to do (seemingly) but unable to perform it. Another week went by, he'd have his eyes open more often, I had him reach out and pick a spiderman figure from my hand one day, he wouldn't do that for another two weeks. The only word he ever really spoke in the office was to his older brother, who brought him a Bumble Bee Mr. Potato Head toy to which he said "thanks". That again, would be the only thing he would say for about the next two weeks.
After that 2-3 week span he would surprise us each day. He had gone from not being able to hold his own head up or even being able to swallow to swallowing and actually LAUGHING. This may sound cliche, but the day I was able to get him to laugh seemed to really trigger a whole new list of things that he was able to do again. He began drinking in very small amounts (he had a feeding tube this whole time) and a week later after ripping out his feeding tube, he ate. And he would not stop eating from there on out until his 15th day of chemotherapy.
So here we are almost two months into this, he's begun his second cycle of his first treatment of chemotherapy (3 cycles in 1 treatment - each cycle is 21 days long). His hair has thinned significantly and he will most likely be completely bald soon. He has already fought off one infection. Just another testament to this little boy's strength. He's most often in good spirits while we are in the hospital, it was a blessing having him home for more than 28 hours (unlike the day he got his infection and we had to take him right back).
We recently welcomed our third child into this world on the 24th of July. So now we don't only have the privilege of taking care of a soon to be 3yr. old and a 5 year old but a newborn to keep us uh..."company" all night long (just kidding the wee one has been great). It's been a new hurdle every few weeks, for instance, just today (Aug. 7th) my oldest son started kindergarten. It will make things easier during the day, but not so much when he needs to be somewhere safe and loving after school. We'll take care of it. We have to!
Wrapping up, I can say that since the time my wife broke down when they told us our sons diagnosis, (with the exception of the choked back fear and anger in the hospital room that one day) I have not had an adequate means of ventilation. I thought perhaps this would help, but it really hasn't. I have not yet told my wife the story about him saying he was scared, I'm afraid it would tear her up too much (she doesn't read this blog, it bores her). Sorry, just had another support person drop off dinner for us. That's another thing, the support network of people we've had has been simply outstanding. Mowing the lawn (it's a big, very steep yard), cooking dinners, doing laundry, shuttling our oldest son places and so on, it's been a great, great help.
If there's one thing I've learned in all of this it's that God won't give me more than I can handle. I am intrigued to see just how far that limit goes as I believe it has been pushed much further than I would have expected. Once we get through this, I feel that my family from our newborn on up will be stronger collectively for it. Especially my little trooper.
I'll follow up this post with information regarding the benefit concert a friend of ours (who just today found out her father was diagnosed with brain cancer) has put together here in Nashville for our son to raise some money for his medical expenses (don't worry, we are insured, but insurance does not cover 100% obviously).
If you read all of this, I applaud you. It's tough to hold onto a though and go with it so I bet this was probably a choppy read...
Thanks again to everyone who has helped out through this and even those of you who just give me that pat on the back when it's clear things are rough. I can't say again how lucky I am to have such a great network of friends, family and even just online acquaintances.
Thank you all.
Brad
Friday, March 6, 2009
The Lantern Project
I've been having a bit of a love affair with various storage platforms over the last year or so. Some notable mentions of late are Tokyo Cabinet and redis. Both are incredibly cool, fast, trivial to interface and best of all can run across multiple machines. I'll post a few pre-built interfaces at the bottom of this post so jump there for their links.
Anyway, so all of that along with my new-found and seemingly unhealthy obsession with Erlang has driven me to, of course, to build my own...
Enter The Lantern Project.
No, there's no cool story-line behind its name, I wish there were. Anyway, so just a few of the "things" driving this currently are going to be (I don't trust anything with bullets, so I'll just delimit a list): RESTful, simple "string" storage (initially - see reasoning below), based on ETS/DETS (initially - see reasoning below) and finally distributable (that will be more or less the last "phase").
Well, for starters, I absolutely love it as an interface to CouchDB and other service-oriented sites as well (twitter, facebook {to an extent - stop inventing your own methods!}). Second, it's easy implement at the client level, we already know it scales, damn near anything can connect to it, etc...etc. This was a no-brainer, I really wish redis would have this, but, trade-offs, I understand...
I had a fun time mulling over this, do you support more complex objects at first? At first, is what I kept coming back to. If this is a learning project, then let's start small and work our way up. Makes sense (to me), but what other data type could you really use to store "anything" (take binary data out of the picture for a moment). I think that by allowing folks to marshal/encode objects as they see fit (JSON,base64,XML,whatever) that makes for a pretty good first pass.
Simple, it's in Erlang and I don't need to really "worry" about formats to get something functional out the door. If my reading is correct, they're based on Judy Arrays which gives you (I know, b-trees do it too) both constant O(1) and logarithmic O(log n)look-ups, NICE! They generally take up less memory, but you do end up utilizing more CPU. Meh, fair enough...
So there ye have it...if you want to peek at my (ahem) "progress" the github project is right here.
Anyway, so all of that along with my new-found and seemingly unhealthy obsession with Erlang has driven me to, of course, to build my own...
Enter The Lantern Project.
No, there's no cool story-line behind its name, I wish there were. Anyway, so just a few of the "things" driving this currently are going to be (I don't trust anything with bullets, so I'll just delimit a list): RESTful, simple "string" storage (initially - see reasoning below), based on ETS/DETS (initially - see reasoning below) and finally distributable (that will be more or less the last "phase").
Why REST?
Well, for starters, I absolutely love it as an interface to CouchDB and other service-oriented sites as well (twitter, facebook {to an extent - stop inventing your own methods!}). Second, it's easy implement at the client level, we already know it scales, damn near anything can connect to it, etc...etc. This was a no-brainer, I really wish redis would have this, but, trade-offs, I understand...
Why just strings at first?
I had a fun time mulling over this, do you support more complex objects at first? At first, is what I kept coming back to. If this is a learning project, then let's start small and work our way up. Makes sense (to me), but what other data type could you really use to store "anything" (take binary data out of the picture for a moment). I think that by allowing folks to marshal/encode objects as they see fit (JSON,base64,XML,whatever) that makes for a pretty good first pass.
Why ETS/DETS?
Simple, it's in Erlang and I don't need to really "worry" about formats to get something functional out the door. If my reading is correct, they're based on Judy Arrays which gives you (I know, b-trees do it too) both constant O(1) and logarithmic O(log n)look-ups, NICE! They generally take up less memory, but you do end up utilizing more CPU. Meh, fair enough...
So there ye have it...if you want to peek at my (ahem) "progress" the github project is right here.
As promised, the gem links for redis/tokyo cabinet
Friday, February 20, 2009
Erlang, you are holding out on me!
So I was playing around with list comprehensions today and found myself looking at permutations of strings when I noticed I wasn't getting "everything" back from the shell.
I'm sure there is a good reason for this, but I'm unclear as to what to search for in order to find an answer. The code I'm talking about is as follows:
which produces:
I'm sure it is just trying to save some horrendous output, but what if for some un-Godly reason I needed it all, how do I turn those [...]'s into more values?
Anyway, just thought this was interesting.
I'm sure there is a good reason for this, but I'm unclear as to what to search for in order to find an answer. The code I'm talking about is as follows:
[ [String1]++[String2] || String1<-"ABCDEFGHIJKLMNOPQRSTUVWXYZ", String2<-"ABCDEFGHIJKLMNOPQRSTUVWXYZ"].
which produces:
["AA","AB","AC","AD","AE","AF","AG","AH","AI","AJ","AK",
"AL","AM","AN","AO","AP","AQ","AR","AS","AT","AU","AV","AW",
"AX","AY","AZ","BA","BB",
[...]|...]
I'm sure it is just trying to save some horrendous output, but what if for some un-Godly reason I needed it all, how do I turn those [...]'s into more values?
Anyway, just thought this was interesting.
Tuesday, February 17, 2009
Why Erlang will run "The Matrix"
Ok, so I had blogged and even somewhat complained about Erlang as an "every day" language. I'm here right now to say that I was dead wrong.
Erlang is simply amazing as I've gone from merely flipping through pages and references online and in books, to fiddling with gen_server off and on and to actually implementing a small facebook application based on Nick Gerakines' erlang_facebook module.
What I found funny about that last bit (the facebook app) is that after seeing an implementation of it in Ruby, it was actually easier to comprehend just what all was going on in the ERLANG version. Yes, I said it, the Erlang was easier to read.
Getting back on track...
After digging deep into getting my head around Erlang, something kind of crept up on me as I went along. Particularly when it came to message passing and hot code swapping...
You ready? Put on your tinfoil hat. Below is my list of 5 examples The Matrix could exist and why it could be written in Erlang.
1. You ever wonder why we can only really "use" less than 10% of our brains? Ever wonder what that other 90-some percent is doing? That's right, passing messages to other "processes" and handling other distributed tasks.
2. When it comes to the hot code loading, this is a good example of how a "Smith" would work.
3. Every wonder why we can’t utilize 100% of our brains? Maybe it’s because the other 90-ish percent is too busy doing distributed tasks in conjunction with other processes.
4. Anything that scales just by adding cores (or nice, warm, fresh humans) and has been sitting somewhat idle in the wings for the last 20 years or so has GOT to be hiding something. I mean c'mon, most of the "love languages" (Ruby, Python, PHP) look at a multi-core system and don't really know what to do with themselves.
5. (And this will be the last one as I'm losing momentum) The system can self-sustain for obscene periods of time, if one had the "available systems" (people) one could die (literally and figuratively) and it really wouldn't matter, you'd have another waiting to take it's place AND you've got around 90% of it's processing power at your disposal.
This has been fun() (I can't help myself, I'm sorry) I hope some of you who come across this add some more to this list in the comments, I think there's a lot of cracks to be made.
Erlang is simply amazing as I've gone from merely flipping through pages and references online and in books, to fiddling with gen_server off and on and to actually implementing a small facebook application based on Nick Gerakines' erlang_facebook module.
What I found funny about that last bit (the facebook app) is that after seeing an implementation of it in Ruby, it was actually easier to comprehend just what all was going on in the ERLANG version. Yes, I said it, the Erlang was easier to read.
Getting back on track...
After digging deep into getting my head around Erlang, something kind of crept up on me as I went along. Particularly when it came to message passing and hot code swapping...
You ready? Put on your tinfoil hat. Below is my list of 5 examples The Matrix could exist and why it could be written in Erlang.
1. You ever wonder why we can only really "use" less than 10% of our brains? Ever wonder what that other 90-some percent is doing? That's right, passing messages to other "processes" and handling other distributed tasks.
2. When it comes to the hot code loading, this is a good example of how a "Smith" would work.
3. Every wonder why we can’t utilize 100% of our brains? Maybe it’s because the other 90-ish percent is too busy doing distributed tasks in conjunction with other processes.
4. Anything that scales just by adding cores (or nice, warm, fresh humans) and has been sitting somewhat idle in the wings for the last 20 years or so has GOT to be hiding something. I mean c'mon, most of the "love languages" (Ruby, Python, PHP) look at a multi-core system and don't really know what to do with themselves.
5. (And this will be the last one as I'm losing momentum) The system can self-sustain for obscene periods of time, if one had the "available systems" (people) one could die (literally and figuratively) and it really wouldn't matter, you'd have another waiting to take it's place AND you've got around 90% of it's processing power at your disposal.
This has been fun() (I can't help myself, I'm sorry) I hope some of you who come across this add some more to this list in the comments, I think there's a lot of cracks to be made.
Wednesday, January 7, 2009
Making RabbitMQ Stew
I just wanted to write this up real quick for anyone who may run into the same problem I did...
I'm working on a project where I need to load some work from a queue for indexing into a Xapian Database (coming from CouchDB). So, RabbitMQ seemed like a good fit long-term for such a task.
Anyway, After getting RabbitMQ installed, I went to start'er up and noticed that it kept failing. So, off to the logs we go...and that is where I found the following:
Mnesia('rabbit@[myhostname]'): ** WARNING ** Mnesia is overloaded: {dump_log, ...
Hmm, well as much as I'd love to be pegged for resources I knew this wasn't the case. So after some googling I had found some suggestions that would squelch the warnings. What really seemed to do the trick was starting up rabbitmq-server with the following parameters: -mnesia dump_log_write_threshold 50000 -mnesia dc_dump_limit 40 (this was suggested from the Million User Comet Application
Now, since I know people don't really enjoy reading the documentation on things, I'll make it easier on you. From the mnesia documentation here is a quick description for the above-mentioned parameters.
and
I'd like to note that I can omit dump_log_write_threshold and still get rabbit up and running, so it may or may not help you out.
Now that I had begun passing these parameters to rabbitmq-server I was now getting a new error in the logs:
=INFO REPORT==== 7-Jan-2009::11:52:45 ===
application: rabbit
exited: {{timeout_waiting_for_tables,
[user,user_vhost,vhost,rabbit_config,listener,durable_routes,
route,reverse_route,durable_exchanges,exchange,
durable_queues,amqqueue]},
{rabbit,start,[normal,[]]}}
type: temporary
Oy, alright - so the tables don't seem to either be there or they just refuse to be created. Let's see if I can drop the mnedia tables that we created when starting the server and start over again.
So, I...
cd /var/lib/rabbitmq/mnesia/
an "ls" revealed two rabbit directories: rabbit/ and rabbit_20090107031256/
rm -rf rabbit/
rm -rf rabbit_20090107031256/
This path...is now clear! Let's start rabbitmq-server up again, but this time, without all the voodoo.
rabbitmq-server -detached
Starts up fine and now if we check our logs, we should see something like:
=INFO REPORT==== 7-Jan-2009::12:40:48 ===
Rolling persister log to "/var/lib/rabbitmq/mnesia/rabbit/rabbit_persister.LOG.previous"
=INFO REPORT==== 7-Jan-2009::12:40:48 ===
started TCP listener on 0.0.0.0:5672
The short of it - whipe your mnesia directories and start over.
I'm working on a project where I need to load some work from a queue for indexing into a Xapian Database (coming from CouchDB). So, RabbitMQ seemed like a good fit long-term for such a task.
Anyway, After getting RabbitMQ installed, I went to start'er up and noticed that it kept failing. So, off to the logs we go...and that is where I found the following:
Mnesia('rabbit@[myhostname]'): ** WARNING ** Mnesia is overloaded: {dump_log, ...
Hmm, well as much as I'd love to be pegged for resources I knew this wasn't the case. So after some googling I had found some suggestions that would squelch the warnings. What really seemed to do the trick was starting up rabbitmq-server with the following parameters: -mnesia dump_log_write_threshold 50000 -mnesia dc_dump_limit 40 (this was suggested from the Million User Comet Application
Now, since I know people don't really enjoy reading the documentation on things, I'll make it easier on you. From the mnesia documentation here is a quick description for the above-mentioned parameters.
-mnesia dc_dump_limit Number. Controls how often disc_copies tables are dumped from memory. Tables are dumped when filesize(Log) > (filesize(Tab)/Dc_dump_limit). Lower values reduces cpu overhead but increases disk space and startup times. The default is 4.
and
-mnesia dump_log_write_threshold Max, where Max is an integer which specifies the maximum number of writes allowed to the transaction log before a new dump of the log is performed. It defaults to 100 log writes.
I'd like to note that I can omit dump_log_write_threshold and still get rabbit up and running, so it may or may not help you out.
Now that I had begun passing these parameters to rabbitmq-server I was now getting a new error in the logs:
=INFO REPORT==== 7-Jan-2009::11:52:45 ===
application: rabbit
exited: {{timeout_waiting_for_tables,
[user,user_vhost,vhost,rabbit_config,listener,durable_routes,
route,reverse_route,durable_exchanges,exchange,
durable_queues,amqqueue]},
{rabbit,start,[normal,[]]}}
type: temporary
Oy, alright - so the tables don't seem to either be there or they just refuse to be created. Let's see if I can drop the mnedia tables that we created when starting the server and start over again.
So, I...
cd /var/lib/rabbitmq/mnesia/
an "ls" revealed two rabbit directories: rabbit/ and rabbit_20090107031256/
rm -rf rabbit/
rm -rf rabbit_20090107031256/
This path...is now clear! Let's start rabbitmq-server up again, but this time, without all the voodoo.
rabbitmq-server -detached
Starts up fine and now if we check our logs, we should see something like:
=INFO REPORT==== 7-Jan-2009::12:40:48 ===
Rolling persister log to "/var/lib/rabbitmq/mnesia/rabbit/rabbit_persister.LOG.previous"
=INFO REPORT==== 7-Jan-2009::12:40:48 ===
started TCP listener on 0.0.0.0:5672
The short of it - whipe your mnesia directories and start over.
Subscribe to:
Posts (Atom)
