Simple math will show you that ((4.9-4)*10-9)*10^15=0; however your computer will tell you that it's 3.55. This is because the computer is approximating 4.9 and so most of the time the error is so small it doesn't matter, but if you add up a bunch of these errors then you get a big error.
I always knew there was this precision error in computers, I just didnt think I would ever run into the problem...well now I have. I now must re-specifying my model because my computer is not smart enough to calculate things precisely. This probably would have happened at some point anyway, but I really wanted to have a working algorithm before I worried about the interpretation of what I'm doing.
Tuesday, March 31, 2009
Monday, March 30, 2009
Billions and Trillions
I was asked by someone today whether a trillion was bigger than a billion.
I'm not really sure what to say about this, but I do believe that person represents 'millions' of others that probably don't know.
I'm not really sure what to say about this, but I do believe that person represents 'millions' of others that probably don't know.
Finally!!
March 30, 2009: After multiple delays (approx 3 months), my Journal article has finally been sent out for other people to read!
Still the first phase as we will probably have to revise some more once we get feedback, but honestly at this point I'm just glad it is being sent out into the world of scholarly criticism. At this point I am not even concerned about the criticism my paper might receive.
But as of now the revising the journal is officially on the back burner.
Still the first phase as we will probably have to revise some more once we get feedback, but honestly at this point I'm just glad it is being sent out into the world of scholarly criticism. At this point I am not even concerned about the criticism my paper might receive.
But as of now the revising the journal is officially on the back burner.
First day of the rest of my week...
I am back from a rather interesting weekend which consisted of me and my brother attending two African gatherings with lots of food. It was good to get out of town for a while and I gladly left my work here at the office but now I am back. I also played a game of basketball for the first time in many years; it was fun even though I did suffer an slight injury. I was hit in the eye so hard my contact fell out, amazingly we were able to recover the contact in the darkness since it was nightime, but I threw it away as soon as I drove home.
I am still having the calculation problem I mentioned earlier and I think it has something to do with the exponential term. For example, take x=1 and y=2 while these numbers are now to far away from each other, the exp(exp(1)) and exp(exp(2)) and 15.15 and 1618.18 respectively.
What I think is going on is that small errors are being magnified by the exponential term. This would mean that this would be one of the 1% of time when matlab is making the error and not me. I will run this though by my adviser with whom I am meeting with this afternoon.
Right now I need to solidify my students exam which can take some time. I need to change the problems, work out solutions to the problems, and then assign point values to the problems. I would like to knock this out in like an hour.
I am still having the calculation problem I mentioned earlier and I think it has something to do with the exponential term. For example, take x=1 and y=2 while these numbers are now to far away from each other, the exp(exp(1)) and exp(exp(2)) and 15.15 and 1618.18 respectively.
What I think is going on is that small errors are being magnified by the exponential term. This would mean that this would be one of the 1% of time when matlab is making the error and not me. I will run this though by my adviser with whom I am meeting with this afternoon.
Right now I need to solidify my students exam which can take some time. I need to change the problems, work out solutions to the problems, and then assign point values to the problems. I would like to knock this out in like an hour.
Saturday, March 28, 2009
Making Moves
Final thoughts on the week:
We won the brain bowl for the 3rd consecutive year; it was pretty fun. We had the BGSA dinner last night and the food was surprisingly good. Supposedly everyone was suppose to go out but I think after all that food most just decided to go to bed.
I was determined to do something, which ended up being TGIF for some more eating and then bed. Once thing that has been frustrating me lately is the over consumption of work by everyone around me; no one seems to take a day or night off anymore. I feel as if I need a little more action around me. Camping or beach trip....something different.
I go my oil changed and put my last revision on the journal paper (yes it did not get sent off on friday); monday is the new sendoff date. I am leaving for Pensacola at some point today and hopefully can make it before the thunderstorm hits.
We won the brain bowl for the 3rd consecutive year; it was pretty fun. We had the BGSA dinner last night and the food was surprisingly good. Supposedly everyone was suppose to go out but I think after all that food most just decided to go to bed.
I was determined to do something, which ended up being TGIF for some more eating and then bed. Once thing that has been frustrating me lately is the over consumption of work by everyone around me; no one seems to take a day or night off anymore. I feel as if I need a little more action around me. Camping or beach trip....something different.
I go my oil changed and put my last revision on the journal paper (yes it did not get sent off on friday); monday is the new sendoff date. I am leaving for Pensacola at some point today and hopefully can make it before the thunderstorm hits.
Friday, March 27, 2009
Man vs. Computer
I decided against re specifying my model after I realized I had a bug in my program that was completely throwing of my estimates. So now everything is working, however I have ran into a very annoying problem that happens every so often. A theoretical result that I absolutely know is true is not being verified by my computer. I have programmed long enough to know that 99% of the time, there is an error in either my logic or what I am programming and not the computer and so the fact that I can't correct the problem is quite frustrating
A simple example of the problem I am facing that I know for a fact that (x-2)^2=x^2-2x+4
Yet when I have my computer compute: (x-2)^2
I get a different value than: x^2-2x+4
Even more frustrating is when I generate my data one way, I get same number in both equations; it's just when I generate the data using another way.
I cannot for the life of me figure out what I am doing wrong, I have been staring at this for hours and almost embarrassed to have to ask my adviser about this. Sometimes when you stare at a problem so long, the simple things become the problems. I really should be quite thrilled about having obtained good estimates of all the parameters, but this is really clouding my high.
A simple example of the problem I am facing that I know for a fact that (x-2)^2=x^2-2x+4
Yet when I have my computer compute: (x-2)^2
I get a different value than: x^2-2x+4
Even more frustrating is when I generate my data one way, I get same number in both equations; it's just when I generate the data using another way.
I cannot for the life of me figure out what I am doing wrong, I have been staring at this for hours and almost embarrassed to have to ask my adviser about this. Sometimes when you stare at a problem so long, the simple things become the problems. I really should be quite thrilled about having obtained good estimates of all the parameters, but this is really clouding my high.
Thursday, March 26, 2009
Still no complete solution
This week will end in suspense over my derivations. My new approach is to respecify the model and see if I can get better estimates, if not I will somehow have to make what I have work in some meaningful way. I feel as I am very close, but yet so far.
I had a couple of issues with my group projects I have been assigning in that 1 group member was not helping out on the project. A slight blame game started but I do believe the situation has been resolved. I gave my students and extension on their project until Tuesday; from what I have seen, I think this new project was pretty successful and so I think I will add this one to the coursework.
The BGSA brain bowl is today and I am a participant so I will attend that at 8 and hopefully tomorrow I will finish the last revision on my journal paper.
I had a couple of issues with my group projects I have been assigning in that 1 group member was not helping out on the project. A slight blame game started but I do believe the situation has been resolved. I gave my students and extension on their project until Tuesday; from what I have seen, I think this new project was pretty successful and so I think I will add this one to the coursework.
The BGSA brain bowl is today and I am a participant so I will attend that at 8 and hopefully tomorrow I will finish the last revision on my journal paper.
The Obama Gamble
Most of my training as a statistician had lead me to view most real life outcomes a probability instead of certainties. Regardless of what I really think will happen, there is a change that what I think is wrong. For me that is a given assumption, what really matters is the probability.
For example suppose you play a game with only two possible outcomes, you either win $1000 or lose $10000. You might play this game if there was at least a 90% chance of winning, but might not if there was only a 60% chance. A statistician would decide whether this game was worth playing by calculating something called the expected value. Basically, if you know the dollar value and probability of each outcomes; you can calculated this quantity. If it is positive you would play the game, if not you wouldn't.
Similarly, Obama is betting that if we borrow and spend enough money on healthcare, energy, and education, we will save money on healthcare in the future, make money from a new energy economy, and a make money from better educated public. There is also chance as in the words of Rush Limbaugh that he fails. As a statistician, I would like to ask the following question in today's townhall meeting:
I would like to know your administration's estimated dollar amount gain assuming you succeed as well as the estimated dollar loss if you fail. I would also like to know your estimated probability of these two events.
Somehow I feel as if the question is a little wonkish, but I think the debate over the estimates the White House provided would create quite alot of chatter in the analytical community.
For example suppose you play a game with only two possible outcomes, you either win $1000 or lose $10000. You might play this game if there was at least a 90% chance of winning, but might not if there was only a 60% chance. A statistician would decide whether this game was worth playing by calculating something called the expected value. Basically, if you know the dollar value and probability of each outcomes; you can calculated this quantity. If it is positive you would play the game, if not you wouldn't.
Similarly, Obama is betting that if we borrow and spend enough money on healthcare, energy, and education, we will save money on healthcare in the future, make money from a new energy economy, and a make money from better educated public. There is also chance as in the words of Rush Limbaugh that he fails. As a statistician, I would like to ask the following question in today's townhall meeting:
I would like to know your administration's estimated dollar amount gain assuming you succeed as well as the estimated dollar loss if you fail. I would also like to know your estimated probability of these two events.
Somehow I feel as if the question is a little wonkish, but I think the debate over the estimates the White House provided would create quite alot of chatter in the analytical community.
Wednesday, March 25, 2009
Crazy Day...
I spent the first part of the day going over typos with my finance adviser for the journal article. By around 12 I think we were both pretty sufficiently bored with the process. I ran the simulations my stats adviser suggested and low and behold, I definitely have a problem of identifiably in the estimate "h".
Now this means I should try to respecify the model to hopefully a system where all the parameters can be estimated. I am annoyed that my grader has not given me back the assignment for my class tomorrow. The students have an exam, and because he couldn't find the time to grade 10 papers, they will not have their assignments back to study for their exam on Tuesday. Sometimes I just prefer to grade my own paper; but being that they provide me with a grader it's hard not to turn down less work for the same amount of pay.
Did a one-hour tutoring session today. I have a pretty good setup right now with this guy I met a couple of months ago. He tutors maths sessions for a living and any students he can't tutor he passes along to me. Saves me the time of having to go out an find business, and the money is not bad.
I need to know what's going to happen with the FPL position really soon, our apt complex is giving us a deal until April 10 if we re-new and how my life is going to turn out is still up in the air.
Now this means I should try to respecify the model to hopefully a system where all the parameters can be estimated. I am annoyed that my grader has not given me back the assignment for my class tomorrow. The students have an exam, and because he couldn't find the time to grade 10 papers, they will not have their assignments back to study for their exam on Tuesday. Sometimes I just prefer to grade my own paper; but being that they provide me with a grader it's hard not to turn down less work for the same amount of pay.
Did a one-hour tutoring session today. I have a pretty good setup right now with this guy I met a couple of months ago. He tutors maths sessions for a living and any students he can't tutor he passes along to me. Saves me the time of having to go out an find business, and the money is not bad.
I need to know what's going to happen with the FPL position really soon, our apt complex is giving us a deal until April 10 if we re-new and how my life is going to turn out is still up in the air.
Tuesday, March 24, 2009
Tuesday's the busiest day
Every Tuesday, on top of teaching my two classes, I have to do one hour of tutoring in the library for all students taking a statistics class. Normally my hour is at 1 however during "exam week" it starts at 2. Being that "exam week" really is so generic I don't know when it really is; I showed up to my hour early today.
I was sortof annoyed because the library is a decent walk from my department and the pollen outside is really getting to me. I definitely need to get some allergy medicine because I am blowing my nose and scratching my eyes constantly.
Apparently I derived some of the estimates wrong as I got very different equations today. There is now another estimate "b" that I am not sure about now; hopefully I can clarify these issues during my meeting this afternoon.
I can tell that the assignment I gave my students today was difficult for them. Most took the entire session to finish it and I think they are sortof confused with the material. Sometime between now and Thursday I need to make Exam 2 which most likely will happen tomorrow. My plan is to leave school by 6 today so I can go for a run.
I was sortof annoyed because the library is a decent walk from my department and the pollen outside is really getting to me. I definitely need to get some allergy medicine because I am blowing my nose and scratching my eyes constantly.
Apparently I derived some of the estimates wrong as I got very different equations today. There is now another estimate "b" that I am not sure about now; hopefully I can clarify these issues during my meeting this afternoon.
I can tell that the assignment I gave my students today was difficult for them. Most took the entire session to finish it and I think they are sortof confused with the material. Sometime between now and Thursday I need to make Exam 2 which most likely will happen tomorrow. My plan is to leave school by 6 today so I can go for a run.
Monday, March 23, 2009
The academic life
Over the weekend I had a conversation with one of my good friends about work in general. We both grew up in the same church and went to the same high school; after that he went the technical trade route and me the university route. He now works in industry on trucks and I work in academia on equations. Academic life is not for everybody, and here I will list the main pro's in con's in my opinion
Pro's: The main benefit of academics life if the unstructured work schedule. There are two main components to my life in academia: Research and Teaching. There tends to be a lot of flexibility in pursuing your own research interests on your own time; also you pretty much prepare your own lectures and agenda for a class semester. Now I have never done this nor would I, but I could probably leave un-announced from work right now, travel the world or do whatever for a month, and come back without my boss asking me where I have been and still receive my bi-weekly pay-check. Nobody is really breathing down your back to see what you are doing
Con's: From the moment I got here, I had 5 years to take a certain number of classes, pass my qualifying exams, and write a dissertation. There is really not set pace or order in which this need to be done, but you have 5 years to get it done. The hardest things to do is properly pace your progress. Those that pace it too slow are often slammed with massive amounts of work at one times; it's the equivalent of having to run twice as fast the second half of a race because you walk the first half. The other extreme is taking on too many things and not being able to stop running when your supposed to be walking. You end up thinking about your work constantly, asking yourself if you have done enough or if you can improve something. Even on vacation, you must bring your laptop for research along with several journal paper to read. It is hard and very annoying when you cannot stop running.
The con's often makes a 9 to 5 job look very appealing, because you know once 5 o'clock hit's, you don't think about your work until 9am the next day. However, I can imagine it might be hard to give up the freedoms that academia brings. In industry every decision made is to make the company more profitable; this can cause employers to be very demanding of employees while they are on the company's dime. So there is no doubt when it comes to freedom from the corporate structure, academic life is a haven like no other.
From our conversation I concluded that there are days when we really do not like what we are doing, but overall we find satisfaction from our work.
Pro's: The main benefit of academics life if the unstructured work schedule. There are two main components to my life in academia: Research and Teaching. There tends to be a lot of flexibility in pursuing your own research interests on your own time; also you pretty much prepare your own lectures and agenda for a class semester. Now I have never done this nor would I, but I could probably leave un-announced from work right now, travel the world or do whatever for a month, and come back without my boss asking me where I have been and still receive my bi-weekly pay-check. Nobody is really breathing down your back to see what you are doing
Con's: From the moment I got here, I had 5 years to take a certain number of classes, pass my qualifying exams, and write a dissertation. There is really not set pace or order in which this need to be done, but you have 5 years to get it done. The hardest things to do is properly pace your progress. Those that pace it too slow are often slammed with massive amounts of work at one times; it's the equivalent of having to run twice as fast the second half of a race because you walk the first half. The other extreme is taking on too many things and not being able to stop running when your supposed to be walking. You end up thinking about your work constantly, asking yourself if you have done enough or if you can improve something. Even on vacation, you must bring your laptop for research along with several journal paper to read. It is hard and very annoying when you cannot stop running.
The con's often makes a 9 to 5 job look very appealing, because you know once 5 o'clock hit's, you don't think about your work until 9am the next day. However, I can imagine it might be hard to give up the freedoms that academia brings. In industry every decision made is to make the company more profitable; this can cause employers to be very demanding of employees while they are on the company's dime. So there is no doubt when it comes to freedom from the corporate structure, academic life is a haven like no other.
From our conversation I concluded that there are days when we really do not like what we are doing, but overall we find satisfaction from our work.
Meetings
My meeting with my stats adviser wasn't really a meeting, instead we agreed to schedule a meeting tomorrow. I did however present my results over the weekend and he suggested a couple of tests I did not think of doing before that will add confidence to my methodology.
I also met with my finance adviser in which we revised my journal article for at least the hundredth time. I have said this many times before, but I think we are almost done with revision; and hopefully by friday this paper will be submitted and that will be a huge relief for me. I think most of what is left is typos and such...at least I hope so. While I think the paper and it's current form looks multiple times better than it did even on the 40 sumthieth revision, I am definitely tired of looking at it and bored with even thinking about it.
I plan on finishing the assignment I will give my students tomorrow and calling it a day; I can finish everything else tomorrow. I would like to go to this comedy/poetry show tonight, but this this BGSA week and we have an event I believe at the same time so I am not sure of my plans as of yet.
I also met with my finance adviser in which we revised my journal article for at least the hundredth time. I have said this many times before, but I think we are almost done with revision; and hopefully by friday this paper will be submitted and that will be a huge relief for me. I think most of what is left is typos and such...at least I hope so. While I think the paper and it's current form looks multiple times better than it did even on the 40 sumthieth revision, I am definitely tired of looking at it and bored with even thinking about it.
I plan on finishing the assignment I will give my students tomorrow and calling it a day; I can finish everything else tomorrow. I would like to go to this comedy/poetry show tonight, but this this BGSA week and we have an event I believe at the same time so I am not sure of my plans as of yet.
Sunday, March 22, 2009
Decent day's work
I ran the first test and the results suggest that 5 of my 7 estimates are working correctly. There are two that do not give me the correct values I would expect, one have never given the correct value even on simpler problems and so I believe that is pretty much an issue of identifiability; I am quite sure I have the correct equation because I have used it successfully before.
So there is really only one estimate (h) I am unsure about because it's a new equation and I am not getting the correct values back. The good news is that I know the direction I am going is the correct way; you always feel as if you are sort of just hanging out there until you can verify your results...you just don't know if what you are doing means anything. But I think I have done a decent amount of work to bring to my adviser on monday, I will ask if he can verify or correct my h calculation; if he gets the same estimate I will recheck my computation again and if it doesnt work I will assume that this parameter is also not identifiable. I will think proceed to the final test; hopefully I can wrap up testing by the end of the week. I feel pretty good though, when my research is looking promising to me I feel more optimistic about it in general since I know what I need to do.
One reason I am in a hurry to get results is because I am attending a conference next month in which I am suppose to give a poster presentation. My current poster presentation isn't really related to the conference theme however this problem that I am working on now is and I think it would be an more interesting contribution. This is however the best case scenario, I won't stress myself out to reach this goal.
In other news, after looking at the academic calendar I realize that I am running out of time and I am doubtful I will be able to cover everything that I usually do in my class. I definitely need to cover hypothesis testing, but it would be really bad if I skipped confidence intervals so I will try to squeeze that lecture in somehow. I also need to assign the final project as well as a presentation day; I think I will schedule their last assignment for when I am at the conference since I can always get a fellow grad student to handle that.
Well, I am satisfied with my progress today and so I think I will take off.
So there is really only one estimate (h) I am unsure about because it's a new equation and I am not getting the correct values back. The good news is that I know the direction I am going is the correct way; you always feel as if you are sort of just hanging out there until you can verify your results...you just don't know if what you are doing means anything. But I think I have done a decent amount of work to bring to my adviser on monday, I will ask if he can verify or correct my h calculation; if he gets the same estimate I will recheck my computation again and if it doesnt work I will assume that this parameter is also not identifiable. I will think proceed to the final test; hopefully I can wrap up testing by the end of the week. I feel pretty good though, when my research is looking promising to me I feel more optimistic about it in general since I know what I need to do.
One reason I am in a hurry to get results is because I am attending a conference next month in which I am suppose to give a poster presentation. My current poster presentation isn't really related to the conference theme however this problem that I am working on now is and I think it would be an more interesting contribution. This is however the best case scenario, I won't stress myself out to reach this goal.
In other news, after looking at the academic calendar I realize that I am running out of time and I am doubtful I will be able to cover everything that I usually do in my class. I definitely need to cover hypothesis testing, but it would be really bad if I skipped confidence intervals so I will try to squeeze that lecture in somehow. I also need to assign the final project as well as a presentation day; I think I will schedule their last assignment for when I am at the conference since I can always get a fellow grad student to handle that.
Well, I am satisfied with my progress today and so I think I will take off.
Saturday, March 21, 2009
Results!
I had a fairly productive date in that I was able to obtain estimates for all the parameters in my model; I briefly ran into a problem of identifiability which is when you aren't sure what inputs give your the output you observed. I will give a simple example of this issue: Suppose I want to find a value of "x" such that 10=2*x, you can clearly see that x=5 is the only possible answer; this system is identifiable. However if I ask you to find the values of "x" and "y" such that 10=x+y, there are many possible values 10=3+7,10=4+6,...etc. This system in not identifiable.
In order to fix this problem I had to respecify my model to give me more information about the estimates. For example, if you had know that y=2*x, then you could find unique values of "x" and "y" such that 10=x+y, but you had to add that extra information. The results look really promising and if proven true, this might be my greatest accomplishment yet. I now need to verify my equations, there will be two steps in this verification process; the first step will be the easiest to implement and the second step might take some time. So how does verification work? Well let's say that you claim to have some tool (which in my case is the equation) that can tell how fast a car is going. One way to verify the tool is to drive the car at a speed that you know, say 60 miles and hour, then use your 'tool' estimate the speed. If the tool gets a value close to 60 like 59 or 61 then it is probably working, if you get a value like 100 or 10 then it is probably not working and you need to fix the tool. So tomorrow we test the tool.
Breakthrough!
Yesterday I experience a small victory...as I have previously mention, I didn't think the problem I have been working on appeared to have a solvable solutions; well after toying around with the derivation yesterday it now appears possible. My adviser seemed quite confident about this fact all along, but I have been doubtful.
I derived one estimate yesterday and in spite of the very complicated looking equations, I can tell that it has a form that makes intuitive sense. As my adviser once told me: "You should try to understand what the equation is saying". For instance the equation, y=2*x means that you have some variable "y" that is twice the value of another variable "x". This gets harder to do when you are looking at equations that are extremely complex, but nonetheless it helps you to understand what exactly it is you have done. Well, tt is almost 12 so I need to get started and see what happens.
I derived one estimate yesterday and in spite of the very complicated looking equations, I can tell that it has a form that makes intuitive sense. As my adviser once told me: "You should try to understand what the equation is saying". For instance the equation, y=2*x means that you have some variable "y" that is twice the value of another variable "x". This gets harder to do when you are looking at equations that are extremely complex, but nonetheless it helps you to understand what exactly it is you have done. Well, tt is almost 12 so I need to get started and see what happens.
Friday, March 20, 2009
Agghhh!!
I have working on these equation all morning and I have reached a point where 90% percent of the mathematics makes sense, but there's this annoying 10% of the math that doesn't. Many times the issue is small enough to ignore and you figure it out after you come up with you solutions, but these issues are bothering me and it will take a significant amount of mathematics and programming to test my results so I would very much like to be on the correct path. I have emailed my adviser and requested an emergency meeting; I would at least like to have something I can work on productively over the weekend.
We had colloquium this morning. The speaker was definitely old-school; no powerpoint or latex presentation....just the projector. Once a statistician get's to a certain point in their career and has established a name for themselves, there are somethings you can get away with that an graduate student just couldn't. For instance, the speaker's presentation was completely handwritten, he openly admitted not really knowing any real application to the work (though he did mention some insurance problem at the beginning of the talk), and that the one simulations he did might not be correct. A perfect example of a theoretical statistician in the ever increasing applied statistician world.
I didn't understand anything he said other than the first slide. He started of by stating mathematically an insurance problem of ensuring you have enough money in the bank to pay for any insurance claims that might occur with a certain level of confidence. The rest of the talk was about manifold, laplace transform, convolutions, and other complex ideas that I haven't seen or barely remember from my days in differential equations. I don't even think he even solved the original problem. Ok, let me get back to work....after checking every news website in the world!
We had colloquium this morning. The speaker was definitely old-school; no powerpoint or latex presentation....just the projector. Once a statistician get's to a certain point in their career and has established a name for themselves, there are somethings you can get away with that an graduate student just couldn't. For instance, the speaker's presentation was completely handwritten, he openly admitted not really knowing any real application to the work (though he did mention some insurance problem at the beginning of the talk), and that the one simulations he did might not be correct. A perfect example of a theoretical statistician in the ever increasing applied statistician world.
I didn't understand anything he said other than the first slide. He started of by stating mathematically an insurance problem of ensuring you have enough money in the bank to pay for any insurance claims that might occur with a certain level of confidence. The rest of the talk was about manifold, laplace transform, convolutions, and other complex ideas that I haven't seen or barely remember from my days in differential equations. I don't even think he even solved the original problem. Ok, let me get back to work....after checking every news website in the world!
Problems and solutions
I had a very productive session with my advisor yesterday. I feel a little more confidence about being able to obtain a solution for the problem I am working on, but I still think I am a long way off. I will try to create and parallel analogy to the problem that I am working on.
Suppose you observe a ball flying through the air and it could have only came from two places; and the first place would have made the ball fly in the air at a different speed than the second place. There is also a certain probability that it came from the first place. Now if you observed many such balls, The E.M. algorithm can be used to estimates the probability of the balls coming from the first place, the speed if it came from the first place, and the speed if it came from the second place. This is a well known problem with well known solutions.
My issues is suppose that the speeds change every day and you observe one ball every days for several days. Can you still estimate the speeds and the probability? I am not so sure because in the earlier problem there were just two speeds to estimate; in this problem you would have to estimate the speeds at each day.
This is just the 1st issue in my problems, it actually becomes harder because supposed you dont' actually observes the balls, you just observed the winds changes that might occur if a ball is flying in the air. Given only the wind measure can you estimate, the speed, probabilities, and where the balls are in the air? Yep, I have alot on my plate.
Suppose you observe a ball flying through the air and it could have only came from two places; and the first place would have made the ball fly in the air at a different speed than the second place. There is also a certain probability that it came from the first place. Now if you observed many such balls, The E.M. algorithm can be used to estimates the probability of the balls coming from the first place, the speed if it came from the first place, and the speed if it came from the second place. This is a well known problem with well known solutions.
My issues is suppose that the speeds change every day and you observe one ball every days for several days. Can you still estimate the speeds and the probability? I am not so sure because in the earlier problem there were just two speeds to estimate; in this problem you would have to estimate the speeds at each day.
This is just the 1st issue in my problems, it actually becomes harder because supposed you dont' actually observes the balls, you just observed the winds changes that might occur if a ball is flying in the air. Given only the wind measure can you estimate, the speed, probabilities, and where the balls are in the air? Yep, I have alot on my plate.
Television
Recently I have been rediscovering the joys of flipping through the channels and watching random shows. This is quite new to my life as I mostly watch cable news (CNN, MSNBC, FOX); and movies and tv shows on my computer. I ran across a show call "manswers" last night and as the name implies gives answers to questions that only a man would ask. The one I found most interesting was: How to kill a bear with your bare hands. The show claims there is a martial art called Hikuta in which this can be achieved if under attack. I always assumed if I was in this situation the best thing to do would be to play dead, however "manswers" has actually got me thinking that I can fight a grizzly bear!
As wanted to watch the Obama's Late Night appearance on Jay Leno, but I was so caught up in "manswers", I missed the first part of it. I did catch him making his quirp about bowling in which he alluded to bowling like he was in the special olympics; the moment he said it I new that was going to be a story.
This is is second, that-wouldof-been-a-funny-joke-with-ya-boys-but-probably-not-as-president moment. The first came when he made the Nancy Regain seance joke. Once again, I think it will be a day story, but this in my opinion will be the last one he gets away with. The next one will become statistically significant enough for the media to create a storyline that Obama is an "insensitive jerk". I got to be honest and say that each times he's done this, he had me laughing pretty hard because it's the kinda of loose talk any guy would say around their house; I mean common who hasn't make a "special-ed" joke in their life. Nevertheless, this is not a storyline his administration would like to waste their time defending.
I watched a little bit of Jim Cramer's marketwatch show; I have mixed feeling about Cramer. I thoroughly enjoyed Jon Stewart exposing Cramer's "expertise" in stock picking for the fraud that it is last week, but I do see the benefit in Cramer making his show interesting for the average Joe. He's throws stuff everywhere, presses buttons that make weird sounds, screams and yells incomprehensible stuff and talks as if he's knows every stock personally...it's really intriguing and you can't help but watch sometimes. I feel as if he is capitalizing on a market of 20-29 year old that all grew up watching sesame street, where that characters manipulated us into learning our ABC's and 123's by making funny noises, singing and acting funny. Cramer is now using the same technique to teach us about stocks. Now to the person that understand the stock market pretty well, his advice is very elementary but I can appreciate the effort he put into dumbing down a dull subject; as I teacher I know that is difficult. I thinks it's the stock picking stuff and his creation of false or just unprovable storylines for why stock went up or down that turn me off, not the education on the terminology in finance he teaches. It also doesn't help that he was a hedge fund manager that use to do all the shennanigans he preaches against now; but as I said, I just have mixed feeling about his work.
In other news, I thought my lecture yesterday went really well, probably the best central limit theorem lecture I have given. I want to go back and change some of the direction I gave on the game I assigned them to make but I don't want to confuse them. I think I will just wait to see what sort of questions I get first.
As wanted to watch the Obama's Late Night appearance on Jay Leno, but I was so caught up in "manswers", I missed the first part of it. I did catch him making his quirp about bowling in which he alluded to bowling like he was in the special olympics; the moment he said it I new that was going to be a story.
This is is second, that-wouldof-been-a-funny-joke-with-ya-boys-but-probably-not-as-president moment. The first came when he made the Nancy Regain seance joke. Once again, I think it will be a day story, but this in my opinion will be the last one he gets away with. The next one will become statistically significant enough for the media to create a storyline that Obama is an "insensitive jerk". I got to be honest and say that each times he's done this, he had me laughing pretty hard because it's the kinda of loose talk any guy would say around their house; I mean common who hasn't make a "special-ed" joke in their life. Nevertheless, this is not a storyline his administration would like to waste their time defending.
I watched a little bit of Jim Cramer's marketwatch show; I have mixed feeling about Cramer. I thoroughly enjoyed Jon Stewart exposing Cramer's "expertise" in stock picking for the fraud that it is last week, but I do see the benefit in Cramer making his show interesting for the average Joe. He's throws stuff everywhere, presses buttons that make weird sounds, screams and yells incomprehensible stuff and talks as if he's knows every stock personally...it's really intriguing and you can't help but watch sometimes. I feel as if he is capitalizing on a market of 20-29 year old that all grew up watching sesame street, where that characters manipulated us into learning our ABC's and 123's by making funny noises, singing and acting funny. Cramer is now using the same technique to teach us about stocks. Now to the person that understand the stock market pretty well, his advice is very elementary but I can appreciate the effort he put into dumbing down a dull subject; as I teacher I know that is difficult. I thinks it's the stock picking stuff and his creation of false or just unprovable storylines for why stock went up or down that turn me off, not the education on the terminology in finance he teaches. It also doesn't help that he was a hedge fund manager that use to do all the shennanigans he preaches against now; but as I said, I just have mixed feeling about his work.
In other news, I thought my lecture yesterday went really well, probably the best central limit theorem lecture I have given. I want to go back and change some of the direction I gave on the game I assigned them to make but I don't want to confuse them. I think I will just wait to see what sort of questions I get first.
Thursday, March 19, 2009
Bailouts
My brother and I were discussing the bonuses paid to AIG executives this morning. I told him I was wasn't necessarily angry at the bonuses paid because it all depends on how good the executives are at what they do. He didn't understand my reasoning until I came up with a sport analogy:
Suppose that the Cleveland Cavaliers get bailed out by the government and decides to caps the all players pay at half a million. This would be a wonderful salary for "95% of Americans", but what about Lebron James? No doubt, there would be other NBA teams willing to pay millions for Lebron and millions more in bonuses to entice Lebron to leave the cavaliers and join their team. Lets supposes that Lebron leaves the Cavaliers, and his ability were essential to the cavaliers success; the cavaliers now suck and began to lose revenue since people don't like to waste money on losers and therefore cannot pay the government back. So basically, in order to keep Lebron in Cleveland, the Cavaliers would have to pay Lebron ridiculous amounts of the tax-payer funded money to stay competitive with other teams that are vying for him. This is all assuming that his skills are so unique that he is not easily replaceable.
And so back to AIG, I am not aware of how valuable these executives skills are. My intuition tells me that they are just ivy league grads or people with wealthy connections with no special talent, they did after all, stupidly insure a bunch of incredibly risky assets. However, in the case that there are some Lebron James equivalents in the AIG executive branch, those bonuses might be essential to keeping him or her from going to an company with a better offer; and quite frankly, if the taxpayers ever want to see this money again, we are going to need some very talented people working at AIG.
In other news, FiveThirtyEight ran a logistic model on Obama's NCAA bracket and concluded that he had a biased towards swing states
Suppose that the Cleveland Cavaliers get bailed out by the government and decides to caps the all players pay at half a million. This would be a wonderful salary for "95% of Americans", but what about Lebron James? No doubt, there would be other NBA teams willing to pay millions for Lebron and millions more in bonuses to entice Lebron to leave the cavaliers and join their team. Lets supposes that Lebron leaves the Cavaliers, and his ability were essential to the cavaliers success; the cavaliers now suck and began to lose revenue since people don't like to waste money on losers and therefore cannot pay the government back. So basically, in order to keep Lebron in Cleveland, the Cavaliers would have to pay Lebron ridiculous amounts of the tax-payer funded money to stay competitive with other teams that are vying for him. This is all assuming that his skills are so unique that he is not easily replaceable.
And so back to AIG, I am not aware of how valuable these executives skills are. My intuition tells me that they are just ivy league grads or people with wealthy connections with no special talent, they did after all, stupidly insure a bunch of incredibly risky assets. However, in the case that there are some Lebron James equivalents in the AIG executive branch, those bonuses might be essential to keeping him or her from going to an company with a better offer; and quite frankly, if the taxpayers ever want to see this money again, we are going to need some very talented people working at AIG.
In other news, FiveThirtyEight ran a logistic model on Obama's NCAA bracket and concluded that he had a biased towards swing states
Wednesday, March 18, 2009
Lecturing
Tomorrow I will give a lecture on the central limit theorem. I have yet to find a way to emphasize the significance of this theorem to my students. I don't blame them for not understanding it as I didn't get it until well into my first year of graduate school.
I like this result because it pretty explains why pollster can "scientifically" predict election results, and justifies the use of surveys, opinion polling, product testing, and other various sampling procedures. I like it even more because it relies on the use of the normal distribution; and while there are many distributions in statistics, my preference is the normal distribution because it is intuitive and widely applicable.
I plan on giving my students a project tomorrow as well. Basically they must construct a profitable gambling game involving rolling dice or tossing coins; a good gambling game should entice the player to want to play over and over again in spite of the odds guaranteed to be against him in the long run. This is a new project that I am trying out and so it will be interesting to see what kind of projects get turned in. I really wish online gambling were legal in the US; sheesh...I wish gambling were not so heavily regulated, statisticians would be self-employed millionaires!
On another note, I am feeling slightly more motivated about my research which is good because I really need to plan out an exit strategy for my thesis. I am also looking forward to attending the sequential monte carlo methods conference next month as SAMSI. This will be the first conference I attend that specifically focuses on my area of research. I am also looking forward to meeting with Arnaud Doucet; after reading so many of his papers and citation while conducting my own research, he has become a celebrity to me in the land of particle filters. We had one correspondence through email in which he told me what I was doing was not right...I was just glad that he responded. But, I have had very good experiences at the conferences that I have attended thus far and hope this continues.
I like this result because it pretty explains why pollster can "scientifically" predict election results, and justifies the use of surveys, opinion polling, product testing, and other various sampling procedures. I like it even more because it relies on the use of the normal distribution; and while there are many distributions in statistics, my preference is the normal distribution because it is intuitive and widely applicable.
I plan on giving my students a project tomorrow as well. Basically they must construct a profitable gambling game involving rolling dice or tossing coins; a good gambling game should entice the player to want to play over and over again in spite of the odds guaranteed to be against him in the long run. This is a new project that I am trying out and so it will be interesting to see what kind of projects get turned in. I really wish online gambling were legal in the US; sheesh...I wish gambling were not so heavily regulated, statisticians would be self-employed millionaires!
On another note, I am feeling slightly more motivated about my research which is good because I really need to plan out an exit strategy for my thesis. I am also looking forward to attending the sequential monte carlo methods conference next month as SAMSI. This will be the first conference I attend that specifically focuses on my area of research. I am also looking forward to meeting with Arnaud Doucet; after reading so many of his papers and citation while conducting my own research, he has become a celebrity to me in the land of particle filters. We had one correspondence through email in which he told me what I was doing was not right...I was just glad that he responded. But, I have had very good experiences at the conferences that I have attended thus far and hope this continues.
Derivations
The guys I was substituting for showed up on crutches to teach today so that saved me at least an hour of time in my day. I met a little bit earlier with my adviser and we re-derived the EM algorithm for the umpteenth time. Once again he stressed the importance of me be able to do it from scratch and so I just forced myself to go through the derivation line by line so that I will be able to do so next time I am in his office...which will probably be tomorrow.
The basic problem is that my current issue looks very similar to a known gaussian mixture parameter estimation problem that is solved using the EM algorithm. However there are a few issues that in my mind will make the problem insolvable using the EM algorithm; because of this I haven't spent much time going through the details of the derivation and I have been trying other routes that have been dead ends. So I am now back to where I started and perhaps I will see something I didn't see before if I just work the problem out.
We talked about my interview at FPL briefly, in summary if I get the job I can graduate with my Phd in the summer, if not I can stick around for a year, publish, travel, and make my dissertation stronger and more marketable for an academic position. I feel this is very reasonable and it is actually making my job search less stressful and giving me more options. I won't know anything for a few week, and I am hoping for the best but I am not to anxious about landing a position yet...next year I will be.
I went to a talk on campus yesterday about the conflicts between science and religion; I didn't find the talk particularly stimulating intellectually but one thing he said stuck out to me. He was talking about how the early Christian scholars dealt with perceived conflicts with science: If something is interpreted to be literally true in scripture, and it conflicts with something that has been shown to be demonstrably true in science, then you must reconsider your interpretation of that scripture.
It struck me as cynical, like acknowledging something that shouldn't be acknowledged because it sounds bad; but I guess that's what we do anyways if you just replace the word science with reason. For instance take the following verses in the New Testament:
"And these sign will follow those who believe . . . In My name . . . they will take up serpents . . ."
—Mark 16:17-18
Some people literally take this to mean that true Christians should be able to handle snakes and not be hurt; however reason leads us to believe that anyone who picks up a poisonous snake will; therefore most Christian's don't interpret this scripture literally.
Anyhow the talk covered conflicts such as heliocentricity and evolution. Another interesting point the speaker made was one that I pretty hypothesized...that ironically the church actually legitimized science during the middle ages as a practical pursuit despite the many conflicts that arose between science and religion.
There is another talk in a couple of days about the origins of the universe which I think might be interesting, so if I have time I will check it out. In the mean time I will go through my EM derivation once more and prepare for my class tomorrow. Once again I have no lecture note prepared and I still don't know if I want to give them a project to work on over the weekend...let me get back to work.
The basic problem is that my current issue looks very similar to a known gaussian mixture parameter estimation problem that is solved using the EM algorithm. However there are a few issues that in my mind will make the problem insolvable using the EM algorithm; because of this I haven't spent much time going through the details of the derivation and I have been trying other routes that have been dead ends. So I am now back to where I started and perhaps I will see something I didn't see before if I just work the problem out.
We talked about my interview at FPL briefly, in summary if I get the job I can graduate with my Phd in the summer, if not I can stick around for a year, publish, travel, and make my dissertation stronger and more marketable for an academic position. I feel this is very reasonable and it is actually making my job search less stressful and giving me more options. I won't know anything for a few week, and I am hoping for the best but I am not to anxious about landing a position yet...next year I will be.
I went to a talk on campus yesterday about the conflicts between science and religion; I didn't find the talk particularly stimulating intellectually but one thing he said stuck out to me. He was talking about how the early Christian scholars dealt with perceived conflicts with science: If something is interpreted to be literally true in scripture, and it conflicts with something that has been shown to be demonstrably true in science, then you must reconsider your interpretation of that scripture.
It struck me as cynical, like acknowledging something that shouldn't be acknowledged because it sounds bad; but I guess that's what we do anyways if you just replace the word science with reason. For instance take the following verses in the New Testament:
"And these sign will follow those who believe . . . In My name . . . they will take up serpents . . ."
—Mark 16:17-18
Some people literally take this to mean that true Christians should be able to handle snakes and not be hurt; however reason leads us to believe that anyone who picks up a poisonous snake will; therefore most Christian's don't interpret this scripture literally.
Anyhow the talk covered conflicts such as heliocentricity and evolution. Another interesting point the speaker made was one that I pretty hypothesized...that ironically the church actually legitimized science during the middle ages as a practical pursuit despite the many conflicts that arose between science and religion.
There is another talk in a couple of days about the origins of the universe which I think might be interesting, so if I have time I will check it out. In the mean time I will go through my EM derivation once more and prepare for my class tomorrow. Once again I have no lecture note prepared and I still don't know if I want to give them a project to work on over the weekend...let me get back to work.
Tuesday, March 17, 2009
Last thoughts of the day
I reran a simulation today and considered that an accomplishment and it gave me a slight thrill that it still worked. It took about an hour to run and if I could just find a time series to apply it do I would be in business.
I am supposed to meet with my adviser tomorrow to rederive the estimates that I ultimately don't think will work; I am also substituting for a friends class tomorrow as he is having an operation. He uses PowerPoint in his lectures, I have always been chalkboard/whiteboard teacher; I know the PowerPoint put me to sleep when I was taking classes and I guess I assume I would do the same. Anyhow it should be interesting to see how the lecture goes when using powerpoint.
I don't see much use for hanging around the office much longer, I guess I will call it a day.
I am supposed to meet with my adviser tomorrow to rederive the estimates that I ultimately don't think will work; I am also substituting for a friends class tomorrow as he is having an operation. He uses PowerPoint in his lectures, I have always been chalkboard/whiteboard teacher; I know the PowerPoint put me to sleep when I was taking classes and I guess I assume I would do the same. Anyhow it should be interesting to see how the lecture goes when using powerpoint.
I don't see much use for hanging around the office much longer, I guess I will call it a day.
Motivation
I have been suffering from an intense lack of motivation as of late. It all started with an acute case of the flu 2 weeks ago and I have not been able to shake it. I have realized several thing about myself throughout the course of writing my dissertation...first, I absolutely hate writing my dissertation. I hate writing for the most part, which is a one purpose of this blog... forcing myself to do write what I am thinking at the current moment.
I am not very good at doing such things, for instance at the moment I feel anxious, I want to be doing something else but I'm not sure what it is I want to be doing; my brain feels unable to think of a direction or angle to tackle on my thesis right now and I feel as if I am unable to be interested in anything for a prolong span of time. I feel as if this exercise should be easy, but I think my brain gets annoyed when I try to prod it for information on how it is feeling, like it's telling me to leave it alone and let it do whatever the hell it is doing right now. I feel as if everything I am saying right now sounds crazy....so back to why I hate writing.
Writing a dissertation is even more frustrating, it's involved explaining complex mathematical symbols coherently in English, and citations....O how I hate citations. I feel as if everything that I avoided doing well in grade school has finally caught up to me, as if I was able to successfully dodge English my whole life since I was so good in Math it never really mattered; and now it has come back with a vengeance.
One thing I hope this blog does overtime is to help me write down my thoughts coherently and efficiently which in turn will reflect well on my dissertation writing. This is only post number 3 so don't expect anything spectacular yet!
I am not very good at doing such things, for instance at the moment I feel anxious, I want to be doing something else but I'm not sure what it is I want to be doing; my brain feels unable to think of a direction or angle to tackle on my thesis right now and I feel as if I am unable to be interested in anything for a prolong span of time. I feel as if this exercise should be easy, but I think my brain gets annoyed when I try to prod it for information on how it is feeling, like it's telling me to leave it alone and let it do whatever the hell it is doing right now. I feel as if everything I am saying right now sounds crazy....so back to why I hate writing.
Writing a dissertation is even more frustrating, it's involved explaining complex mathematical symbols coherently in English, and citations....O how I hate citations. I feel as if everything that I avoided doing well in grade school has finally caught up to me, as if I was able to successfully dodge English my whole life since I was so good in Math it never really mattered; and now it has come back with a vengeance.
One thing I hope this blog does overtime is to help me write down my thoughts coherently and efficiently which in turn will reflect well on my dissertation writing. This is only post number 3 so don't expect anything spectacular yet!
Monday, March 16, 2009
After the Break
So today is the Monday after spring break and there a couple of goals that need to be accomplished by the days end.
I teach tomorrow and I have not finished my lesson plan; I think halfway through the semester I am not a motivated to create great lectures for my class. I always start off the semester with new examples, homework problems and projects; towards the end I just recycle my old material.
I am supposed to give a five minute talk today to a visiting professor to highlight the work that I am doing in the department; this has turned out to be quite a challenge in and of itself. When I defended my prospectus, I have about 60 powerpoint slides since I had basically a whole hour to talk. Last month, I presented my work again a a conference in which I only had 15 minutes and so I had to chop off about 40 some slides and condense my talk to the mains themes. So now that I only have 5 minutes, I had a chop another 15 slides of of the condensed version; every slides seems very important at that point and I feel it has been chopped down so much it will no longer make sense.
I spent most of my day rederiving a couple of estimates for the model I have been working on; the equations make sense to me however I have very little confidence that my ultimate estimation goal will be feasible from the current direction I am pursuing. So what is it that I do exactly? Let me try to make up a simplest example I can think of:
Suppose you get paid every Friday and your pay this week is a certain percent increase from the amount you made last week. Now lets also suppose that you put a certain percentage of your paycheck in church every week. Now, suppose that I know the amount of money that you put in church every week last year; using only that information can I figure out how much you got paid every Friday last year, the percent increase in you paycheck every week, and what percent of you money you put in church?
These are the types of problems that I work on, and what I dissertation is about. Now I must get back to work
I teach tomorrow and I have not finished my lesson plan; I think halfway through the semester I am not a motivated to create great lectures for my class. I always start off the semester with new examples, homework problems and projects; towards the end I just recycle my old material.
I am supposed to give a five minute talk today to a visiting professor to highlight the work that I am doing in the department; this has turned out to be quite a challenge in and of itself. When I defended my prospectus, I have about 60 powerpoint slides since I had basically a whole hour to talk. Last month, I presented my work again a a conference in which I only had 15 minutes and so I had to chop off about 40 some slides and condense my talk to the mains themes. So now that I only have 5 minutes, I had a chop another 15 slides of of the condensed version; every slides seems very important at that point and I feel it has been chopped down so much it will no longer make sense.
I spent most of my day rederiving a couple of estimates for the model I have been working on; the equations make sense to me however I have very little confidence that my ultimate estimation goal will be feasible from the current direction I am pursuing. So what is it that I do exactly? Let me try to make up a simplest example I can think of:
Suppose you get paid every Friday and your pay this week is a certain percent increase from the amount you made last week. Now lets also suppose that you put a certain percentage of your paycheck in church every week. Now, suppose that I know the amount of money that you put in church every week last year; using only that information can I figure out how much you got paid every Friday last year, the percent increase in you paycheck every week, and what percent of you money you put in church?
These are the types of problems that I work on, and what I dissertation is about. Now I must get back to work
Sunday, March 15, 2009
First Entry
This will be my second attempt at blogging consistently. The first time I successfully did this was in 2005 the summer before I went to grad. school. Four year later, I am done with my coursework, passed my qualifying exams, and am making substantial progress on the dissertation.
Before I entered the program I used to wonder if I would ever "feel" like a statistician....not that I really knew what one was supposed to feel like but I guess now I do feel as if I have decent command of many of the basics concepts. It's a very interesting sort of knowledge because it is easy to forget that the concepts that now are very intuitive to me took months of training to develop and you cannot assume that the average person will just understand why you view the world a certain way.
The central limit theorem for example is something that just makes sense to me. The idea that you can take trials of individuals from some unknown random process, average them and have a known random process is a natural idea that I find valuable, useful and quite interesting. It explains many things that I observe in life and I often reach conclusion on topics like politics and religion based on statistical concepts that are now natural to me; I guess I have developed the mind of a statistician. Perhaps a glimpse into that reality will arise in this blog.
Before I entered the program I used to wonder if I would ever "feel" like a statistician....not that I really knew what one was supposed to feel like but I guess now I do feel as if I have decent command of many of the basics concepts. It's a very interesting sort of knowledge because it is easy to forget that the concepts that now are very intuitive to me took months of training to develop and you cannot assume that the average person will just understand why you view the world a certain way.
The central limit theorem for example is something that just makes sense to me. The idea that you can take trials of individuals from some unknown random process, average them and have a known random process is a natural idea that I find valuable, useful and quite interesting. It explains many things that I observe in life and I often reach conclusion on topics like politics and religion based on statistical concepts that are now natural to me; I guess I have developed the mind of a statistician. Perhaps a glimpse into that reality will arise in this blog.
Subscribe to:
Posts (Atom)