Austen.Ito.Blog

A smart plan

2011-01-01T22:20:00.001-10:00

Just when I thought my goals for 2011 were set, I listened to the Smart Passive Income podcast. Pat Flynn talked about having SMART goals, that is specific, measurable, attainable, relevant, and time bound goals. After taking a look at what I wrote in my goals for 2011, I realize that I didn't really think though what I want to accomplish.

Learn Django
Learning Django is some vague goal that I might get done. Who knows when and how well I'll get I'll do it. So to make a realistic goal, I'll change my goal to run through the entire Writing your first Django app tutorial. If I want to continue along Django, I can add another goal to this list.

Improve my online presence
Vague goal #2. What? How am I going to improve my online presence? I just did a quick search on "Improving my online presence" and I see the same things:

write a blog
contribute to open source projects
participate in online help forums, i.e. Stack Overflow

I already have some technical goals in regards to open source projects and hacking, but blogging is once again a weak point. So how about I quantify my blogging? I want to have 18 posts by the end of this year not including this post. That's 1.5 posts a month. That's sounds like such a small number of posts, but I bet by the end of year I'll be struggling to get this goal in. A nice timely, achievable goal.

Personal Finance
Using cash only for non-recurring purchases is a great goal. I like that goal a lot. I'm already on my way to saving. I also wanted to quantify my savings goals. I need to put a monetary number on how much I want to save. My emergency fund is a weak point I need to address. I want to have $5000 by the end of 2011. With what I have saved, that turns out to be $400 a month. Ouch. A lofty goal, but I worked it out and I can get it done. Totally attainable.

I also have an addendum to my goals. These are things that I've thought of in the past week.

Do the marathon
I'm going to be in the 1% that does the marathon in their lifetime. Training starts at the end of January!

Sky Diving on my birthday
I'm going to make it happen!

iPhone Hacking
I've spent some time working on an iphone application that's really fun. My brain hurts, but I'm going to continue working on it. This part is measurable with the next item.

Average 7 hours a week on something I'm passionate about
Whether it be iPhone hacking or blogging, I want to average 7 hours a week doing something I love. 7 hours a week isn't that much. It's one hour a night. Totally achievable. I'm going to keep a log on google docs that keeps track of what I'm doing and when. Reading blogs and things online doesn't count because I'm only consuming information. I want to be a producer, one night at a time.

Phew. Now that sounds like a goal list. Time to get to work!

Personal Update: The end of 2010

2010-12-12T07:27:00.000-10:00

One of the great things about having a blog is being able to look back at what you decided to do about your life and and measure how far you've come. I dusted off my previous update, 2010 Things to Get Done to determine if I executed on my action item list. And with that, here is how I did.

Financial Goals - Total, complete, epic fail. Well almost. I didn't stick to my budget, but I did manage to save for a couple of vacations.

Paying Myself First - Going to the gym in the morning before work lasted for about 4 months. Then I had a week of going to sleep late and that killed the early morning gym fests. I still go to the gym 5 times a week, but they are at the end of my day instead of the start.

Read One Library Book A Month - Sigh. Fail again.

Play with Python - Nailed it. I learned a bunch of Python by playing with the YouTube Rest API and the last.fm API.

Complete 2 Resume Updates - Half credit. I did one.

Increase StackOverflow Reputation to 1000 - Fail. I got to 172. I also stopped frequenting SO unless I needed a question answered.

Rails Development - Failed. I did some Rails hacking this year, but I didn't deploy my app. I also canceled the domain I bought to deploy the app.

How depressing. My action item score is:

Completed: 1
Half Credit: 2
Failed: 4

On the surface, it appears that I had a horrible year. I only completed one thing that I set out to do, made some progress on others, and totally dropped the ball on the rest. But to prepare for this post, I did a resume update and found that the majority of my time was dedicated to work:

Worked on a bunch of interesting things like: GWT, Client/Server architectures, Performance improvements.
Mentored an intern during the summer. I learned a lot from him and I hopefully taught him a bunch of stuff.
Used a bunch of frameworks, technologies and tools like: ActiveMQ, Camel, JMS, Tomcat, Derby, Git, Mercurial

Some things that I've learned and along the way, but aren't resume material are:

Making a conscious effort to listening to my team's suggestions and recommendations
Revisiting my software design and ideas the next day to double-check if it was actually a good idea
Being more open to new ways of accomplishing tasks
Read world news in the NY Times in addition to reading a lot of personal improvement, finance, and technology blogs.
I've become an avid yelper at aus.yelp.com

So maybe I didn't do so bad this year. I may not have accomplished my action items I set at the beginning of the year, but I definitely didn't stagnate. I feel like I should have done more, but at least I made some progress.

So where do I go from here?

This year I'm going to try something new. I'm going to create two lists, an "Action item" list and a "Maybe do it this year" list.

Action Items

Learn Django - To continue my software learning, I want to learn some Django. I've done some Python work so now it's time to move my learning to the web. I'm thinking of looking for an OSS Python web based project to contribute to.

Improve my online presence - My online image needs to improve. I need to contribute to online technology forums, work on my linked in profile, and anything else that will increase my visibility.

Personal Finance - My yearly struggle with sticking to budget has made the action item list once again. This time I'm going to try paying with cash for everything except for gas and my monthly bills that are automatically charged. It's sort of like the Envelope System, but I'm not going to bother with sorting my cash into physical envelopes.

Read More Books - I really love reading. I just need to get off my butt and use the library more.

Visit Japan - I need to get to the homeland. In order to go, I'm going to have to save, save, save.

The Maybe List

Take a side job - I really want to try working a part-time job on the side for fun. Something like working at Starbucks or at a wine store. I'm not sure it fits into my work schedule, but I have been toying with the idea.

Andddd go!

Checkstyle: It's Important

2010-04-02T00:55:00.000-10:00

People have a love/hate relationship with Checkstyle. Some people, myself included, are in the love camp. We include it in our development process. We strive for 0 checkstyle warnings. We fear committing Checkstyle warnings because of the ensuing shame resulting from Hudson unstable build emails. Other people tend to hate checkstyle. They think that checkstyle piles on work on the already piled on work stack.

Here's why that's the wrong mindset to have.

Checkstyle enforces your team's agreed upon code style guidelines

Code style is very religious subject. Very Religious. It creates just as many holy wars as the "My IDE is better than yours" argument. But I think everyone can agree that having code style guidelines is a good thing. It aids teams by keeping code uniform and easy to read. Reading code is hard enough without having to weed through 20 different ways types of if-statements and for-loops.

This one is easy. You have style guidelines, let's make sure that everyone is following them. It doesn't take much work to integrate Checkstyle into the daily build. Enforcing code style is a good thing. It really doesn't matter what the style is. What matters is that everyone follows what rules the team has agreed upon.

Checkstyle forces you to write documentation

Developers hate writing documentation. If you love writing code comments then you are a sick, sick person and you probably already love Checkstyle. Feel free to stop reading. Now for the rest of us, we often have a personal two-way connection to our source. We know what it does. At least we know what it does right now. What about two weeks from now? One month? One year?

Integrating Checkstyle into your development process forces you to stop and think. It makes you question what your code does. It can help validate your design. Often times I find myself asking questions like:

What exactly does this method do?

What are it's invariants?

Are their any special cases?

Can I pass bogus values?

What does this algorithm accomplish?

The intricacies of your code exist in your head at this very moment. Why not write it down? Checkstyle makes it so.

The argument against forcing developers to write documentation is that they will write useless comments. This isn't a fault of Checkstyle, it's a fault of the developer. Why aren't they writing good documentation? Why aren't other developers getting on the bad documenter's case? Is anyone else reviewing their code to make sure they aren't being sloppy? Checkstyle doesn't do any of that. Your team does.

Checkstyle isn't for you, it's for everyone else

I believe we all can agree that clear, concise, and well documented code helps other developers. There is a feeling of pure joy when you use a well-documented third party library. Since all code you write will be used by other developers, shouldn't you impose the same documentation standards on yourself that you wish everyone else would follow?

What about the argument of "Why should I care about checkstyle? I know how to document". Even if it were true that you always wrote great documentation, 99.9% of us aren't like you. We need help verifying that we are indeed following code style standards. Checkstyle keeps us honest. After all, we are developers and developers are lazy. Sometimes we need a little prodding to keep ourselves disciplined.

Checkstyle is just a tool. If you don't use the tool correctly, then of course it's going to suck. Used in the right way it's extremely powerful, even for something as seemingly trivial as code style.

Developer Responsibilities

2010-03-27T01:11:00.002-10:00

I think Eric Sink got it right.

We Need Developers, Not Programmers.

In the article, he writes about needing developers who know how to be flexible and understand the software lifecycle instead of coders that are one trick ponies. You know who I'm talking about. Those programmers who know how to write some source and nothing else. Improve the build environment? Work on continuos integration? Unit test? Write wikis to help the team? Mentor? Contribute to the awesome company culture? Forget about it.

Working for a small company, I completely agree with that statement. I'm glad that I work at a company that allows me to have a hand in everything. My tasks during the week range from development environment scripting to application deployment to middleware hacking to pretty front end stuff to ensuring code quality to maintaining our build server. Then I get to write down what I've done to help out the rest of the team on our company wiki. My hard work is rewarded with my #1 status on the Continuos Integration Game leaderboard. Having my hand in everything is awesome because

I'm learning tons everyday!

As a developer, that's something to strive for. Especially if you're a developer that's fresh out of school. When you get out of college, you want to absorb as much as possible. Can you do that at a big company? Maybe. Is it easier at a smaller company? Totally.

As a developer, you have a mound of responsibilities other than whipping out lines of source. Specializing in something is great, but being an agile developer is much more useful to your peers. If you understand the different aspects of software, you can apply your knowledge to challenges you face in every facet of development. I've learned over the years that just being familiar with what people are talking is HUGE. It gives you a basis when you start learning new stuff on The Google. The only way to get to that nice and comfy "I've heard of that before" place is to dive into as many new things as possible.

Staying agile is important and a small company helps you do that. It helps you and your team get things done.

And isn't that what it's all about?

Solving problems, improving customers lives, and rocking out to an awesome product.

Hotkeys. More productivity, More Awesome.

2010-03-04T22:30:00.002-10:00

At a recent career fair, Aaron Kagawa asked students the question,

"What is your favorite hotkey in Eclipse?"

Sadly, not very many students had an answer. Joel Lazaro, a student in Dr. Philip Johnson's 414 class, is on the right track and posted a useful link on Eclipse shortcuts.

Being a productive developer means you trim the number of steps do your hacker tasks. Why is this important? Let's take a look at an extreme example, a daily task in Eclipse with and without hotkeys:

A simple task with hotkeys

Ctrl-Shift-T to open Foo Class
Ctrl-O to find and move to another method
Write some code
Ctrl-Shift-Up/Down to move to the next method
Ctrl-D to delete a line
Control-F11 to run the code

Now the same task without Hotkeys

Grab the mouse
Open the package navigator and search for the package where Foo Class exists
Double-click to open the Foo Class
Scroll with the mouse to find the method to edit
Write some code
Grab the mouse
Move to the next method
Using the mouse, highlight the entire line to delete
Move the mouse to the 'Run Application' Button
Run the code

The main tasks to notice in the 'Without Hotkeys' example is the time spent searching and moving your hand to the mouse. When a developer skips out on the hotkeys, productivity suffers. Too much time is spent searching. When the file to edit is located, more time is spent searching for the place to edit. Once editing is done, it's back to the mouse to move someplace else. Rinse and repeat. *Yawn*

That isn't even the worst part. The worst part is the productivity of the entire team suffers. Less things get done because time is wasted. When other developers try to help, even more of their time evaporates. Let's take a pair programming example. It could be a formal session or an impromptu 'walk-by' to help someone out. The navigator stands their idling while the driver opens a class or finds a method using the mouse. Now the productivity of two people has flatlined. If you're the driver, you're job is to drive fast.

To better explain why hotkeys are so important, I'll reference the age-old analogy of a carpenter and his tools. Think of a carpenter's toolbelt. Pure genius right? It's inventor thought, "I'll wrap a piece of leather around carpenter's waist so they can grab their tools quickly rather than waste their time going back to their tool box." Productivity went through the roof. Carpenters pick the right tool without moving or looking down. Commence carpenter flow time.

Hotkeys help developers to get "in the zone". When you stop to find a class or method, there are more opportunities to break your train of thought. Moving your hands off the keyboard and onto the mouse is like switching contexts. Switching tasks is bad.

Are you ready to improve another skill in your toolset? Take a look at Developers favorite Eclipse hotkeys or hit Ctrl-Shift-L in Eclipse to list the available hot keys. If you use another IDE, take some time to learn about it's hotkeys, macros, and shortcuts.

Still don't believe me? Think using the mouse is ok?

Steve Yegge says, "Non-touch-typists have to make sacrifices in order to sustain their productivity"

Death to the mouse!

Getting Technically Started

2010-02-13T11:58:00.013-10:00

Where do I begin?

This question scared the crap out of me when I forayed into the job space straight out of the university. Everything was wild and unknown. I needed a tour guide yesterday.

As a new grad, I tried a lot of things. I read books and blogs. Listened to podcasts. Worked on open source projects like Hackystat and personal projects like an Open Microblogging Plaform and a ping pong score tracker. I tried a bunch of different approaches to improving my technical life. This blog's aim is to help you filter out the noise. I'll write what worked for me with the hope that it can bootstrap your improvement outside of college.

First things first

There are two articles are forever part of my hacker livelihood. Having read these two articles in Philip Johnson's software engineering class helped me define who I am and to understand what I need to do to contribute. These two webpages. Yes I said webpages not blogs. It's crazy how old these articles are. Notice the manual revision history at the top? Crazy. Nonetheless, the content is the stuff of legends.

How to Become a Hacker

Defines who we are and how we get there. We are hackers. Be proud and proclaim it.

How to Ask Smart Questions

Asking dumb questions makes you dead weight. Hackers hate dead weight. We want to spend time having fun chatting about the latest tech news, making paper airplanes, having Nerf gun fights, and if it's in the project plan, write some code. We don't have the time or patience to answer stupid questions. Remember the "There is no such thing as a stupid question" line doesn't apply to our world. There are stupid questions and it makes us angry.

With that said, we geeks get it. No one knows the answer to everything. It's not humanly possible. Robots, ninjas and zombies can do it, but not us lowly humans. Geeks genuinely want to help people. They want to explain a topic they're familiar with not only flex their intellectual guns, but to help you out.

Ask a smart question, get a smart answer. Respect +1.

Ask a dumb question, lose respect. Respect -100

Respect is king amongst the Geeky.

Blogs

Blogs are an excellent knowledge resource. StackOverflow has a thread on the best programming and development blogs if you are interested, but here are the blogs that I've been following for quite some time:

Basil Vandegriend: Professional Software Development
Coding Horror
Creating Passionate Users - It's a shame that this blog shutdown do someone stalking Kathy Sierra. It's still worth it to go through the blog archives.
Joel on Software
Rands In Repose
reddit.com: Joel on Software - what's new online - Kinda like a hacker version of digg
Steve Yegge's Blog Rants
Hacker News - Up to date tech news
Kohsuke Kawaguchi's Blog - The lead of Hudson. This guy does some crazy stuff.
Slashdot - Geek news for geeky people like you and me
xkcd.com - This isn't a blog, but every hacker should read xkcd. Seriously.

I also follow the blogs of people I've met at the university or through work. They are all interesting as well.

Engineering Log - The maintainer of the jupiter-eclipse-plugin
HackyHawaii For Life by AK - Hasn't been posting lately, but great advice for new grads.
PMJ Engineering Log - Philip Johnson's blog.
Seize the Moment of Excited Curiosity for the Acquisition of Knowledge
SidSavara.com - A personal improvement/life hacker blog. Cool dude, cool blog.

Reading Code

As a hacker you want to get into the nitty gritty. You want to jump into that source and show it who's boss. You're going to write millions of lines of unit tested, bug free, elegant code. But wait.

Do you enjoy reading other people's code? No? That's too bad because reading code is about 90% of your hacker life. Maybe more. There's API's, javadocs, code examples, and source digging among the countless other ways to read lines of code. Rockstar programmers that perform 10x better than others rock at reading code. For the rockstars, reading code should be like reading a book. It needs to flow through your mind and paint a picture of what's happening. It's said that you need to practice 10,000 hours to master something. I think you'll need even more to master code reading because everyone writes their code differently and most of it is bad. You'll need to be able to quickly pick up favored patterns and apply it to your overall understanding of the code structure and usage.

It might seem obvious, but you should get out there and read code. It's actually really easy:

Step 1: Find an interesting project on SourceForge or Github.

Step 2: Check out the source

Step 3: Read the source.

Voila. You are on your way to code reading mastery. You could even take a step in your code writing mastery hour allotment by contributing to the project.

Writing Code

I don't need to write much here. Practice, practice, practice. I would suggest immediately starting with a new language because it promotes new ways of thinking. You'll realize the importance of learning new languages once you apply a language design pattern from one language to another.

I started with Ruby on Rails because I was interested in web frameworks. I've made a resolution to work with Python this year. Pick something you find interesting and do some Code Kata or create yourself a side project.

Blogging

Blogging is an excellent tool. I know that it's not for everyone, but I find that it continually improves my thinking and writing skills. I brainstorm during my commute to work, the walk to Starbucks, the lonely period in the bathroom. I think about interesting blog topics and if my readers (all 5 of them) would get it. Blogging puts your thoughts into written form. Sometimes it's hard to express what you mean until you try to explain it to someone else.

Another great thing about blogging is that it can help you with your Periodic Resume Updates. You won't be able to remember what you've done over the past year, but having an activity log will help figure out what you should add to your resume or tell you that you aren't investing in yourself enough.

Hang out with Hackers

The last point I have to make isn't really a skill, but rather something to help build on your hackerism. Find smart hackers and hang out with them. In person would be best, whether it be your coworkers or people at other software companies. It could even be online. It's similar to pair programming without any coding. Either of you can drive, but both learn from each other. Most geeks like to talk about geeky things like the latest and greatest tools. Find people that are on the bleeding edge and attach yourself to them. The bleeding edge is where you always want to be.

Purpose. It's that little flame.

2010-02-01T09:00:00.000-10:00

Purpose from the Broadway musical Avenue Q. Awesome.

When I first started out writing this blog, I wanted it to be a place where I talked about the happenings in my technical life. A good start I think. Sometime this past week, three years into blogging on and off, I noticed something about the blogs that I read. I'm so dense. They all provide information that help me in some way. Whether its explaining how I use a specific technique to improve my career, an interesting idea, or a novel concept, they all exist to lend a hand.

I noticed that a lot of my posts are a bunch of rants on what projects I'm working on, my love-hate relationship with certain technologies, or my personal improvement progress. Maybe it's time to put a spin on what I blog. Maybe I should write posts about how activities on projects flick on the lightbulb or create guides for tools I'm using. Even reviews of tools and books would be helpful.

Maybe I shouldn't care what others think about my blog because part of the reason for taking the time to write things down is to get my thoughts out of my head. If I really cared about increasing my readership, there are a lot of other things I could do. Tons of people reading my blog isn't my purpose.

Helping others improve is a noble purpose. It will be that little flame.

2010 Things to Get Done

2010-01-08T20:31:00.000-10:00

In the spirit of new years I decided to do what everyone else is doing, create a list of things I want to accomplish this year, i.e., my New Year's Resolution. I remember I created a List of Things I Want to Do way back in 2008. Let's see how I matched up 2 years later:

Improve my writing skills
Ruby/Apache Wicket and Hackystat
Practice Puzzles
Play with Python
Give a speech to a group about something technology

The things I didn't complete are in red. How sad. Two years later I still haven't tried out Python. I think I have improved my writing skills a bit, but I had a blog hiatus in 2009. Hopefully we will miss that episode this year. I tried implementing a Hackystat UI using Ruby on Rails in 2008. Other than mentoring a student for Google Summer of Code 2009, I haven't done any new work on Hackystat. The Collaborative Software Development Laboratory has started new work in the smart grid energy field. It's an interesting area with mounds of real-world application potential. I'm excited to see the direction they take their research. Perhaps I might find an opportunity to contribute to their work.

I didn't create a list of things to get done in 2009, but I did have a Periodic Resume Update that talks about what I've done and where I'm headed. It seems that I believed that Openmbp was the direction I was headed in 2009. That idea has passed and now it's onto new things. I'm a year older and starting to think about new things.

In the past two years I was concerned with everything technology related. I wanted to improve my hacking skills and learn new technologies. Learning new technical skills is important, but I now believe that other aspects of life that should receive attention. That's what I want to accomplish this year in addition to continuing my technological advancement pilgrimage. I debated on creating a list by priority, but I decided that everything on this list should get done. I ended up categorizing the items into financial, personal, and technological goals.

Financial Things to Get Done

Stick to my budget - I'm a failure at sticking to budgets. I spend enormous amounts of time tracking where my money is going, set up a budget, and then buy a digital camera. Or a snowboard. Or a wine fridge. Or some other thing that I probably should have saved up for first. My new strategy is paying myself financially and to use cash when possible. Paying myself first financially means that I put money away before I start paying my bills or buying frivolous things. Using cash helps me to be conscious of where my money is going. Credit cards are evil and they own me. (Hopefully not this year)
Save 10% of income - Paying myself first will allow me to accomplish the next two goals. There are things I want to save up for in the future. I can't purchase them unless I start saving now. I think 10% is a reasonable number.
Save 5% of income for life's experiences - For the past three years I've never saved money to spend on vacations or things I'm passionate about. I was in the financial bias of saving as much as possible and would feel guilty if I decided to take a trip or spend some cash on something I took pleasure in. No more! Over the past year I've become extremely interested in personal finance. So like I always do when I want to find out more on a subject, I look for reading material, i.e. online blogs. A recurring theme that I found that struck a cord with me was: Spend money on things you value. After all, money is just a tool to live life. So this year I resolve to save money for that snowboarding trip or a trek to foreign country. I'm going to save and spend with no remorse. I'm going to live this time around.

Personal Things to Get Done

Continue to pay myself first - I recently wrote a blog paying myself first. I want to continue to go to the gym in the morning.
Read one library book a month - I've been using the library for about a 1.5 months now and I'm going to say that I can't imagine my life without it. I paid 10 dollars for a new library card and the first books I checked out were worth at least twice the library card fee! I've found that I won't have the time to read, but I'm saving money instead of wasting it. I can always renew books I failed to read or just return them and check them out later. The goal here is to continue to read. One book a month is totally doable.

Technical Things To Get Done

Play with Python (Really) - It's time to get off my ass off my crappy chair (My home chair. My work chair rocks!) and learn a new language. I wouldn't say I'm stagnating since I'm still absorbing Ruby and Rails, but I need to get some new code perspectives into my brain this year. Python will be welcomed.
Complete 2 resume updates this year - The periodic resume updates are a good indicator of my current technical and personal development progress. Stagnation equals Fail.
Increase my stackoverflow reputation to 1000! - I think I'm the most excited about this thing to get done. I've been frequenting Stack Overflow(SO) for the past couple of weeks and have decided to join the game. I want to raise my reputation points to a certain level before the end of the year. Since it's a game with rules and points, my progress can be measured. I want to have a goal of 1000 reputation points before 2011. I'm at 96 right now. An idea I had to use SO as a learning tool was to try to answer questions in unfamiliar areas. For example, I'm not a pro Rails hacker, but I want to be. So my task is to try to answer Rails questions even if they already have answers. At the end of the day, reputation points don't matter, what I learn does.
Practice developing with Ruby on Rails - It's still here. Rails hacking. It's fun, but I really need to get a move on. I have a side project that I wanted to deploy by the end of 2009, but I failed. Let's shoot for the end of 2010.

Phew. That's a lot to accomplish. Onward!

My Mockito Tech Talk. AKA: Conveying Information and Stuff

2009-12-21T08:43:00.000-10:00

It seems that 3 weeks after writing Paying Myself First, I have already failed. Well not completely.

I haven't been writing down my thoughts in blog form because I've been preparing slides for a tech talk at work. I've been wanting to give a talk about Mockito. My project is on a transition point so I figured that it would be imperative to give a talk this month before we ramp up again in January.

I suck at public speaking. That process of reading your audience. Conveying information. All of that stuff. Who needs it right? Hook me up with my IDE, give me a fridge full of diet mountain dew and get out of my way. I got things to do.

If only the world worked that way.

Since I'm on this whole, let's work hard and do awesome kick, I've been looking at public speaking as an area to focus on. After all, if I want to be able to lead software teams, I need to be able to be someone who is able to be informative and also know what the hell they are talking about. I've done the whole Toastmasters thing before. I like the concept, but I didn't like the structure and organization of the particular group I was a part of. So without any outside help, I got to searching on how I could improve my slides and effectively present a technical presentation to my peers.

And the search began.

I found this interesting PDF using the Google: Presentation Tips - How to Give a Technical Presentation. It's main points were to break up your presentation into distinct beginning, middle, and end sections. The normal presentation layout, which I totally brainfarted on. My presentation sections were a mess. I didn't show any importance, background, or motivation in the beginning. I had no illustrations to help my audience during the meat of my presentation. The end was the best part, most likely because the slides came to end.

When I fixed my slides a bit, I changed my presentation's beginning to grab my audience's attention with some facts about my project's level of code coverage. The middle gave the basic functionality of Mockito such as mocking, stubbing and verifying. I added some code examples and gave some background on when and why you would want to use Mockito's features. There were a lot of questions that were brought up during this part of the talk. Some were of mocking and stubbing use-cases and others were related to project specific usages. I got a sense of confusion from the audience at this point. I tried to use the List examples from the Mockito documentation, but that turned out to be a bit too high-level for a presentation. Most of the developers didn't "get it".

I finished off my presentation with a "real-world" Mockito example where I tested the behavior of a action class populating a stack. The action class implemented ActionListener and would operate on Stack's object interface. I got good responses from this part of the presentation. I think that developers had that "ah hah" moment when they could finally see Mockito in action from an IDE.

Overall I think the talk went well. It was very informal and the I was getting a good vibe. The best part was the fun-ness factor. I'd have to give it a 9 out of 10. This was the most fun I have ever had during a tech talk. One thing I noticed that might have increased the fun-ness factor was the room's size. Due to scheduling conflicts we had to switch to another room where the developers were comfortably crammed together. Best move ever. I think that helped the talk out a lot because it made the presentation feel less like a presentation and more like a discussion. I think I'll always have my talks in that room if it's possible.

My team has already started to use Mockito to write their unit tests. The real measure of success will be if the rest of the project's teams will start to drink some of that delicious Mockito Koolaid. Oh yeah. A coworker on another project likes the idea of Mockito and believe that he will use it for his tests. Something interesting I ended up telling him was that:

"If you already write tests, then you know the pain of trying to setup and build the dependencies of the object you are testing. If you don't write tests then you'll be wondering what's the point".

I hope they get it.

Baselines, Branching and Reviews!

2009-12-03T08:35:00.001-10:00

We have come to a crossroads at work. On the left we have our latest release. On the right, we have the road leading to future changes, improvements, and success!

Our latest release, lets call it FixLaterReleaseNow or FLRN for short, mainly consisted of developers committing to the trunk. This caused developers to commit unfinished, broken code to the mainline. When other developers would update, they would pull the broken code into their local workspace. I'm not sure if this led to the code rot that manifested itself, but something was wrong. Very very wrong.

Perhaps there was a problem with our peer reviews. We had little to none. The only set of eyes looking at the code were the author's. In FLRN we used the Jupiter Eclipse Plugin to facilitate our peer reviews. Jupiter is a great tool, but there were two major sticking points for us:

Some developers use IDEs other than Eclipse.
The turnover from setup -> moderation -> fixing review issues was way too long. On average it probably took developers 1.5-2 days to complete the process.

I love Jupiter. I've been talking with Julie, the current maintainer of Jupiter, about how we can improve the plugin. That talk is for another blog on another day.

There were broken windows everywhere and no clean up crew to whip it into shape. Something needed to change and fast. We had no time in the FLRN codebase, but we have some time (a little) in the new release to get moving towards the proverbial Awesome Path.

To get on track we have integrated Crucible, a code review tool from the great people at Atlassian, into our development process. It's a bit buggy and doesn't integrate with the other Atlassian products as we would like, but Crucible increases our review turnover. It is much quicker to choose which files to review, comment on the source, and complete the review. The reviews are persisted so other people can checkout what happened in the review. What we are noticing is that developers are posting comments on lines of code and the author comes in and comments on his design decision. You just don't get that kind of immediate feedback when using Jupiter.

So back to the developers committing broken code to the trunk. We don't want that. We want code that has been reviewed by other developers who are working on similar feature sets to be committed to the trunk. So my question is "How do we setup our branches?". (Keep in mind that we use Subversion)

We are currently thinking of having two lines. The trunk, which contains "stable" code that developers can branch off of. In this context, stable means that the feature is complete and somewhat tested. There will be bugs, but we will iterate through them as we go.

The second line will be our feature branch. The feature branch can have many other branches contained in it. This is where developers can work on their specific features without touching the main line. In FLRN (Fix Later Release Now in case you forgot) developers would wait on committing their features because they were broken, or there would be a lot of rippling changes throughout the system, or because they wouldn't have any documentation which would cause our build to become unstable. Now if we force developers to use the "Feature Branch" they can commit at any time without fear of breaking anyone else.

Once the developers are ready to merge to the trunk, we can have other developers review their code. This seems like a good idea. Developers can commit early and often without breaking anyone. They can get their code reviewed by their peers. Everyone is happy.

Perhaps in the make believe place of No Bugs Land.

Complications arise when more than one developer is touching the same file. What if two developers finish their features but are using the same file. Do we merge their changes? Can we combine their features into a separate branch and then create a review? That might work.

We definately need some review and merging management. The problem we want to avoid is having a backlog of reviews. For example, Developer A might be working on file X. He finishes and requests a review for it. Later that day Developer B finishes his feature which includes file X. He requests a review for his code. We now have a problem because there are two revisions of file X to review. If we don't merge both Developer A and Developer B's changes, we will have to review Developer A's changes, merge the changes, then review Developer B's changes. I feel the Horribleness!

I'm wondering how people handle different code baselines in their development environments. I'm going to have to do a bit of research, but this article by Jeff Atwood is a good starting point: Coding Horror: Software Branching

(Edit: It seems that Jeff Atwood got his information from Streamed Lines: Branching Patterns for Software Development)

I am Japanese. Use the Library.

2009-11-19T08:07:00.003-10:00

Do you remember the old Hawaii State Public Library Commericial where kids would say?

I am Hawaiian Chinese. I am Hawaiian Portuguese. Please use the library.

Well I'm not Hawaiian, but I've decided that I'm going to give the library a chance. Up until this point, I told myself that I don't want to use the library. It's such a hassle. Waste of time, energy, resources, and space.

I've been reading in personal finance blogs like the The Simple Dollar about the benefits of using the library. A few of my friends have started or are already using the library. One of even suggested I help to improve the library's borrow/renewal web UI. An interesting side project.

But I had my doubts.

I know they don't carry the latest technology books
Driving to and from the library sucks
Will they have the book I'm looking for?
I really like to write in my books
I'm one of those people that have to have to book just in case there is that one time I _really_ want to find something in a book I bought 5 years ago.
I bet the hours are inconvenient.

So before I even went down to the library I checked their hours: Hawaii Public State Library Hour of Operation

Woah. They have decent hours. They are even open till 8 PM on some days.

So I headed down one day after work.

Here is what I found:

If my book is at another branch, I can submit a request online to ship the book to the branch of my choosing.
It really doesn't take that much time to drive to the library.
The selection isn't the greatest.
They still don't carry the latest technology books.
I still can't write in the books.

I was quite pleased with my library visit. I had to purchase a new library card because I must have thrown my old card away. When was I ever going to use the library again? So that was $10. It was funny because I had a .90 fine... from 2001. It's been 8 years and they still kept a record of returning a book late. Damn them.

But it's okay. I spent $10.90 on my library visit, but I get to have an unlimited amount of books. The money I save from not having to buy books definitely out weighs the time it takes to drive to and from the library. If I reserve my books online I can further reduce the time I spend at the library. Not to mention that I'm saving space because I don't have to store the books. After years of buying books, I notice that my room is cluttered with books that I've read once.

Even if the library doesn't have the technical books that will keep me up to date, I still can purchase those books online at Amazon. I add highlighting, notes to my tech books so it's not a huge deal to buy them. I'm thinking I should try to buy them used instead of new. I don't need the new, clean version of the book. The content is the same.

The library is awesome!

It's too bad they don't have any funding. When I checked out my books, the librarian handed me a flier that told me about their furlough days. On the other side of the flier there was an article about a fundraiser to support the library. They are obviously in a financial bind.

I'm not sure how to help them. I think donations would help the problem, but would it fix it? Maybe if more people used the library? Who knows, but I'm hoping the library never goes away.

Paying Myself First

2009-11-14T10:58:00.000-10:00

Wow. My last post was back in May. It's been a while since I've made time for this blog thing, but it's been a good sabbatical. I'm reemerging on the interweb for the third time and hopefully here to stay. I want to stay committed and vigilant, as we all know how difficult it is to be an active blogger. My plan of posting twice a week turned into posting once a week, which then turned into not posting. Plan failure. But I have a new and improved plan!

One day I had a 6 AM gym session with a friend. It changed my life. My schedule used to look like this:

9:00 AM Wake up
9:05 AM Go to Work
7:00 PM Read blogs, Side Project Work
10:00 PM Hit the Gym

By the time I started working on my personal improvement tasks I would be tired and unfocused. I was amazed at how one morning gym session would make the rest of my day was incredible. I felt great. I didn't need to have 6 cups of coffee or some sugar to get through the day. My metabolism was in overdrive. I couldn't believe how awesome it was to start your day off with a workout.

So I changed my schedule:

5:45 AM Hit the Gym
7:45 AM Read blogs, Learn some cool stuff
7:00 PM Watch video blogs, listen to podcasts, take a break
10:30 PM Sleep

Then one morning I found a blog that talked about Paying Yourself First. What an excellent idea. I've always struggled with spending too much at the beginning of the month or when I get my paycheck. This is a great idea. Pay myself first.

Light Bulb.

My updated scheduled:

5:45 AM Hit the Gym
7:45 AM Write a blog
7:00 PM Read blogs, Learn some cool stuff, Work on side projects, Watch video blogs, Listen to podcasts, Take a Break
10:30 PM Sleep

So my new goal is to do some writing before I head to work. I don't have to write a complete blog or always try to finish existing blog. My goal is to jot down my ideas. If I can write down what I'm thinking about I should be able to get a blog out every week.

I honestly rather do my side project hacking in the morning when I'm the freshest, but I don't have enough time between my gym session and when I need to leave for the job. Once the tasking at work calms down a bit, I should be able to resume my Rails hacking.

A while ago I read an article by Sid Savara, Paying Myself First With My Time. I didn't buy into it then. How could I wake up early and start my day like that? Crazy Talk.

_Best Idea Ever_.

Book Progress

2009-05-25T16:47:00.002-10:00

I enjoyed reading Clean Code. The best part about it was it changed my outlook on method and class construction. My code is _a lot_ cleaner. The last chapter in the book, "Smells and Heuristics", is something all developers should read. I plan on giving it to developers on my company's engineering team.

Next on the reading list is the GoF book. It's interesting to come back to a book years later. A quick scan through the patterns shows me that I have forgotten a bunch. I can't wait to get through it again. The GoF book is great. I'll write more as a I plow through it.

Also Google Summer of Code is ramping up. Time to get to work!

Cleaning up my code

2009-05-03T14:02:00.000-10:00

I've been reading Clean Code that I listed on my list books to get through. I've read a bunch of code improvement books, but there were a couple chapters that I've gone through that I liked so far.

Writing clear functions

We've all heard that we should write functions that only do one thing. But how many of us really do it? Here is an example of how I used to write code.

public void mouseClicked(MouseEvent event) {

// Let's do something on a primary button double-click
if (event.getButton() == MouseEvent.BUTTON1 && event.getClickCount() == 2) {
// do something

}

Inline comments are evil. If they aren't updated, which can happen frequently, why not write a method that can't lie. What I should be doing is writing methods with clear names that inform developers of my intent.

Inline comments can be out of date, in the wrong place, or just plain wrong. Method names get updated automatically if you refactor your code and if the method name is wrong, the compiler let's you know.

Here is how the code should look:

public void mouseClicked(MouseEvent event) {

if (this.isPrimaryButtonDoubleClick(event)) {
// do something
}

}

public boolean isPrimaryButtonDoubleClick(MouseEvent event){

return event.getButton() == MouseEvent.BUTTON1 && event.getClickCount() == 2;

}

The code is trivial, but when your methods and classes get larger, readability and intent become increasingly more important. The longer developers have to stop and think about what your code is doing, the more it needs to be refactored. Here is what a developer might be thinking when they read my old code example:

"We want to do something when BUTTON1 is pressed and we click twice". "What is MouseEvent.BUTTON1"? *Stop to read some java docs* "So we get the click count twice. Oh thats a double-click!"

You might think that someone should immediately associate the clause to a double-click, but you might have to stop and think if you aren't familiar with the author's style or are just getting into Java. Context switching causes breaks in concentration. Not good.

In the refactored code, the developer can get to mouseClicked() and read once, "We want to do something when the user's primary button is double-clicked". No context switching, no breaks in concentration. Good.

Writing clean comments

The next thing I read that was interesting was method level comments. Here is an example of my old method level comments that I would write to pass Checkstyle and document what my method exposes:

/**
* Returns true if the primary button is double-clicked.
* @param event the event wrapping the mouse state information
* @return true if the primary button is clicked, false if not.
*/
public void isPrimaryButtonDoubleClick(MouseEvent event){

return event.getButton() == MouseEvent.BUTTON1 && event.getClickCount() == 2;

}

There are three areas of duplication. First, my initial comment duplicates what the method name states. Second, the @return tag says the same thing. Finally, the method signature clearly states what my method does and returns. Since checkstyle requires a @return tag, here is my refactored method comments:

/**
* @param event the event wrapping the mouse state information.
* @return true if the primary button is clicked, false if not.
*/
public boolean isPrimaryButtonDoubleClick(MouseEvent event){

return event.getButton() == MouseEvent.BUTTON1 && event.getClickCount() == 2;

}

I removed the initial comment to reduce the comment duplication. If the method had no parameters, the code will be reduced further:

/** @return true if the primary button is clicked, false if not. */
public boolean isPrimaryButtonDoubleClick(){

return this.event.getButton() == MouseEvent.BUTTON1 && this.event.getClickCount() == 2;

}

It passes checkstyle and reduces the amount of comment duplication. I like it.

Hackystat + Ivy = Win?

2009-04-22T22:38:00.008-10:00

It's Google Summer of Code time so I checked out Hackystat and tried to build it from source. A couple hours later, my Hackystat development environment was setup. Ouch.

The hardest part about setting up my environment were all of the tool dependencies. To build

hackystat-utilities, hackystat-sensorbase-uh, hackystat-sensorshell, I needed to the following libraries:

Then I needed to set the following environmental variables:

HACKYSTAT_UTILITIES_HOME
HACKYSTAT_SENSORBASE_HOME
JUNIT_HOME
FINDBUGSHOME
APACHEJCSHOME
APACHECOMMONSLOGGINGHOME
RESTLETHOME
JAVAMAILHOME
DERBYHOME
CHECKSTYLEHOME
EMMAHOME
HACKYSTATANTSENSORSHOME
ANTARGS
PMDHOME

As a developer it would be cool if I could check out the source files and run a couple of ant targets to retrieve the dependencies.

At work I've been experimenting with Ivy to download my project's dependencies. I download the dependencies to the project's library directory via an Ivy ant task and I'm done. There is no need to configure any environmental variables because the build script knows the "Home Folder" of the external libraries.

I checked on the Hackydev Mailing List and found a thread started by Aaron a long time ago called Ivy for Java hackystat dependency management.

I think it would be really useful if it were easiser for new developers to check out the source and start hacking. Integrating Ivy into Hackystat's build system might be a useful project and help to reduce Hackystat's hacking entry barrier.

Book Reading Queue

2009-04-19T12:50:00.002-10:00

I've been slacking on the blogging and the reading so to get motivated I'm posting a book queue.

My goal is to get back to reading a book a month. I've read the GoF book already, but it's time to refresh my design patterns. I read it over 3 years ago, so I will probably have a new understanding of when/where to use certain patterns.

I've also changed my reading to include fun and non-technical books. Hopefully that will keep me reading. I've noticed that it's a drag to read technical books all the time. The books are interesting, but my mind needs something new every now and then.

Openmbp groups

2008-11-16T22:58:00.002-10:00

"Hashtags are a community-driven convention for adding additional context and metadata to your tweets. They're like tags on Flickr, only added inline to your post. You create a hashtag simply by prefixing a word with a hash symbol: #hashtag." - http://twitter.pbwiki.com/Hashtags

This past week we spent working out the kinks of our in-house testing of openmbp's basic features like posting, following, favorites, account information. We've got a bug tracker up and soon it will be filled with tons of things to work on. With the in-house test continuing for a while, I'm turning my attention to post grouping.

Twitter is going to role out groups sometime in the future and we are going to start working on getting grouping into openmbp. We are probably going to use hashtags to mark if a topic should be categorized.

Periodic Resume Updates

2008-10-19T22:55:00.003-10:00

Rands says that you should update your resume every 6 months regardless if you are looking for a new job or not. I haven't updated my resume in a couple of years, so this was a pretty interesting exercise. Updating your resume let's you figure out what you've done and where you are heading. So that's what I did tonight. Let's see what I learned...

What I did:

I spent a lot of time working on Hackystat. This is a good thing since I learned about Java, Ant, XML, JAXB, Postgres, and SQL.

I had one line about my work project. I'm not sure if that's a good thing, but I have learned quite a bit from my work experience.

I worked A LOT on the software development platform at work. My work experience section for my job is mostly about setting up out continious integration build server, working on our Ant build scripts, code reviews and mentoring interns. I was debating if I should put some information on trying to start a Wiki usage movement at work, but I decided against it since it didn't really pan out.

I was actually surprised that I had enough "Volunteering" items to put on my resume. I have to thank Aaron for that one.

Where I'm going:

This one was a tough one. I'm looking at my resume and it isn't clear where I'm headed. It seems that I'm interested in hacking for fun. Maybe mentor students and help developers grow or something. Looking at my resume I don't get that "ah hah" that's where he's headed.

But there's hope!

Even though OpenMBP isn't on my resume, I'm pretty sure it's the direction I'm headed. I want to build software that's useful to people. I want them to enjoy using software that I've been working on. Development on that project is starting to ramp up so I hopefully in the next resume revision I'll be able to add a new OpenMBP resume item with Ruby on Rails, Git, and an URL to the released version of our software. I think that's a good goal to have.

I really feel that I haven't done enough in the past two years. Before I started updating my resume, I was thinking that I would have lots of things to add. It turns out that I didn't really add all that much. I added some things about my work project, extra activities like Hackystat and volunteering, but it all felt a bit lacking. I couldn't add anything about the books and blogs I'm reading, the blogs I'm writing. I guess resumes are a way for companies to weed through people and find who they really want to interview. It looks like I'm going to have to work harder in the next six months to put things on my reusme that make me stand out.

Revolutionary Improvement perhaps?

Counter-Blog: Path to Awesomeness

2008-10-16T22:57:00.002-10:00

Jason is an intern at the company I work for and he wrote his take on the Path to Awesomeness. As a mentor, I hope to be helping him along his awesome journey. I decided to write a counter-blog that talks about how I'm helping him along the way.

Learn bout real projects: Requirements, customers, deadlines. Ah the joy of real projects. The classroom environment tries to provide these things, but it doesn't compare to working on a real project. We put our interns on existing projects that will expose them to all of stresses and fun times of a real world project. Right now Jason is working on a real world project and doing quite well.
Learn about research: The first day that Jason started, I gave him an article on How To Ask Questions The Smart Way. Since that day he has never asked me another dumb question ;) But as Aaron pointed out in his comment, the research skill that we want to impress on our interns is learning how to research. My goal is to expose Jason to a wide variety of interesting science topics which may spark some interest and maybe some gradute research of his own.
Learn about software development processes: I'm concentraing very hard in this area to teach him about the entire software process. On his journey, I want to him to be fluent in software tools such as Eclipse, JUnit, Checkstyle, Findbugs, and other static analysis tools. I'm focusing on his code quality and implementation with frequent code reviews using Jupiter. I'm trying to get him to start off with good habits by commiting early and often. He got familiar with Hudson, our continous integration build server, by adding his own module to the integration build. My goal is for Jason to absorb as much as possible by working closely with the developers on the project team.
Learn how to learn: Figuring out problems on your own is a very important skill to have. As everyone knows, Google is your friend. This might be the most important skill for to learn because it will help you throughout your entire career. Basically, learning how to learn means trying to figure out a problem on your own. If you still don't understand what is going on, think of some educated questions and ask your peers. They will be more than happy to help if you have invested some time on the problem. This item is a must for all interns to understand. Luckily, Jason picked this one up on his first day.
Learn how to increase marketability: Jason wasn't too sure on this item. We want our interns to be wanted by companies in the high-tech industry. Our goal is to increase the skills of our interns for their benefit rather than our own company interests. By increasing our interns value, they in turn increase our company's value. Marketability means having something other companies want, like your l33t skillz. We want everyone to see that you outperform everyone else at the same level. We want you to stand out and be awesome amongst your peers. How we want to accomplish this is by exposing our interns to as wide array of work experiences that they can't get in school, but also focusing on key areas that we believe will help them succeed and stand out.
Learn about the industry: Finally, we want to our interns to know what's out there. What other types of work is out their in the government and private sectors. By showing them what type of work is available, they can make a choice on their career path. Hopefully that career path involves our company, but we want our interns to enjoy their career. We want to help them succeed in their career and life even if that means saying goodbye.

An interesting perspective on Kaizen

2008-09-14T12:13:00.004-10:00

A couple months back at Java One, I bought a book called Think Better, An Innovator's Guide To Productive Thinking, and forgot about it somewhere in the clutter space that is my bedroom. I spent this past week nursing an allergic reaction at home so I decided to pop open the book and start reading. The author Tim Hurson is a thought provoking author giving some excellent examples people living constantly in "reproductive" thinking mode. Reproductive thinking mode is the that mode our brain goes into to save energy. We are all guilty of this... A trivial example he gives is waking up in the morning. You don't think when you wake up in the morning. If your mornings are like mine you:

Wake up.
Brush your teeth with the same brush strokes.
Put on clothes from the same subset of work clothes.
Get in the car and drive the same way to work.

But have you ever thought about the "why" behind everything you do? Why do you always brush your teeth in the same direction? Why do you always put a shirt on first before your pants? Why cofffee? Why not tea? I've simplied the examples, but you can see this translating into other aspects of your life. Why do you always solve problems in the same way? On one hand it might be because you are using tried and true patterns that solve the problem in the most efficient way possible. On the other, you could be locking yourself into the same routine:

"Things went on like that for awhile - prison life consists of routine, and then more routine. Every so often, Andy would show up with fresh bruises. The Sisters kept at him - sometimes he was able to fight 'em off, sometimes not. And that's how it went for Andy - that was his routine. I do believe those first two years were the worst for him, and I also believe that if things had gone on that way, this place would have got the best of him" - Red, Shawshank Redemption

I've only read through half of the book, but Think Better attempts to show you the thinking prison that we are living in. It's showing why the same routine that let's us feel comfortable will only lead us to living on the same curve.

All very interesting material. Thinking is good, routine is bad. Then I came up to a chapter titled, "Kaizen vs. Tenkaizen". As you probably know, especially fellow hacker Ryan K, Kaizen is a Japanese philiosophy that focuses on continous improvment though all aspects of life. Tim Hurson argues that Kaizen is not always the best philiosophy to live by. Continous improvement is a great thing if you want to keep chugging along on the same level. But Tenkaizen, which is a word he made up, means "Good Revolution". Producing new innovative ideas, rather than reproducing the same old ideas is what can take us to the next level.

A presentation that comes to mind is Guy Kawasaki's talk about the ice delievery company, let's call them Quick Ice. Quick Ice believes in the Kaizen system. They want to continously improve their ice delievery for their customers. So they work on continously improving. They buy faster trucks, send out larger delieveries, develop efficient ways to produce ice, and waste less ice during deliveries. All of these ideas can come through the philiosophy of Kaizen. You are continually building upon what you already know. But the real innovation occurs when someone has the genius idea of letting customers make their own ice! A refrigerator! Genius! Definitely a product of productive rather than reproductive thinking. Productive thinking lets you jump to that elusive next curve.

I'm very interested in the rest of the book. It has already made an impression on me by making me think about the "why" instead of being a drone and always doing what I've always done. I've decided that I'm not gonna let routine get the best of me.

DailyProjectData Queries are hard and complicated

2008-08-20T20:34:00.004-10:00

I've been spending my night time internet time working on SQL queries that will return records with the DailyProjectData (DPD) information conforming to the DPD REST API. It's been a pretty rough road just getting the queries to work correctly. Now that I have it working, I have to work on the performance of the queries. I know that I wrote a query that performs badly, but just getting it to work was pretty tough since I'm a noob with SQL.

Here is the Code Issue Query that I wrote that returns the amount of code issues for all tools between a specific timeframe for all data. If you want to get the data for a specific project, there needs to be a sensordata.resource clause in the query. The records that are returned have columns with the type of code issue, the latest runtime, tool, and the code issue total:

SELECT issue_type, tool_runtime, tool, sum(cast(issue_count as integer)) from (

-- This sub query gets all of the records without grouping the issue counts
SELECT sensordata_properties.key AS issue_type, sensordata.runtime
AS tool_runtime, sensordata.tool, value AS issue_count FROM "sensordata"
INNER JOIN sensordata_properties ON (sensordata_properties.sensordata_id = sensordata.id) WHERE (sdt_id = (select id FROM SensorDataType where name ~* 'CodeIssue')
AND tstamp > '2008-08-012T00:00:00.000' AND tstamp < '2008-08-13T00:00:00' AND sensordata_properties.key LIKE 'Type_%' AND sensordata.runtime IN (

-- This subquery gets the latest runtime
SELECT latest_runtime FROM (select max(sensordata.runtime) AS latest_runtime, sensordata.tool FROM "sensordata" INNER JOIN sensordata_properties
ON (sensordata_properties.sensordata_id = sensordata.id) WHERE (sdt_id = (select id
FROM SensorDataType WHERE name ~* 'CodeIssue') AND tstamp > '2008-08-012T00:00:00.000' AND tstamp < '2008-08-13T00:00:00' AND sensordata_properties.key LIKE 'Type_%') GROUP BY tool) AS latest_runtime_query))

GROUP BY sensordata.tool, runtime, sensordata_properties.key, value ORDER BY key) AS issue_sum_query GROUP BY issue_type, tool_runtime, tool ORDER BY issue_type

Phew. Thats a big query. I need to figure out how to refactor the query so I don't repeat the same sub query twice. These queries are currently being using in rails with the find_by_sql method, but James told me to check out the :includes directive because it is very powerful and can handle what I'm trying to do without using specific SQL

The good news is that our DPD service implementation is compatible with the Wicket ProjectBrowser that the CSDL team is working on. Right now we can retrieve the raw SensorData and Build DPD summaries. Hopefully, I'll be able to get the rest of DPD queries working and we can start using the ProjectBrowser at work.

Postgres DbImplementation

2008-06-28T16:39:00.002-10:00

I think it's been about 2 months, but I've finally got a working, distributable, Postgres Sensorbase Implementation up. If you are interested, you can check out the discussion on the hackystat-dev mailing list.

The distribution process turned out to be quite interesting because Aaron and I went through a few iterations on how to package up the binaries without having the user to do too much. My original idea was to bundle up the sensorbase.jar classes into one big sensorbase-postgres.jar file. Aaron ended up hating this idea during a chat session of night-time internet (what we call after work fun learning time) .

The first reason why bundling jars up is that you are stuck using whatever libraries are included. If there is a new feature in a later release you either have to 1. Wait till the authors release a new version with the included libraries or 2. Re-jar it yourself. If the libraries aren't bundled you have the flexibility to swap the jars out. Hopefully the interfaces haven't changed or else it won't work anyways.

The second reason is much worse. If libraries are bundled together and they use different versions of the same dependent library, which version is loaded onto the classpath? Who knows. If all depenent libraries are not bundled in jar files, you can't have the class loader clobbering problem because there will only be one version of each library.

In the end we decided that the binaries of the distribution should consist of 2 files. The first is the sensorbase.jar and the second is the sensorbase-postgres.jar. The sensorbase-postgres.jar file contains a very minimal set of classes that are used as a library to the sensorbar.jar file. Starting up the sensorbase with the Postgres implementation is as easy as starting up the original sensorbase.

java -jar sensorbase.jar

Text editors are not the answer

2008-05-28T21:03:00.005-10:00

Invest the time in mastering development tools, on the other hand, and you won’t want to give these tools up to try a new language —- the development tools are a major part of your productivity. For you, a new language doesn’t have many advantages over other languages anyway, because you haven’t studied how to use language features to make you productive.

Josh recently linked a blog post called, The IDE Divide, which made me start ranting to everyone I could find online. If you read it you can clearly see that Oliver Steele is an ultra-biased language maven. I don't understand how he can put people into 2 buckets, either you learn languages feature and are considered a pro-hacker or you use IDEs and suck. He honestly believes in using text editors because you can code anything. I could code using a typewriter and achieve the same effect.

In his diagrams, he shows that you have higher productivity gains if you use a text editor and learn language features rather than learning an IDE and not as much language features. I have two issues with that:

IDEs don't code for you, unless you use Visual Studio (says Josh). You still have to learn the intricacies of the language. Choosing an IDE doesn't lock you into a language. Not having awesome tools may introduce a larger barrier for entry, but it by doesn't stop people from learning new things. IDEs are just tools that help you get your job done.
What happens when you master an IDE? Do you just sit there and twiddle your thumbs? Mastering an IDE doesn't take nearly as long as mastering a language. Even if you spent all your time up front learning an IDE, there would still be a lot of time to learn the language. In fact, you don't even need to master an IDE to master a language. Maybe you could use the IDE as just a text editor with some language specific help thrown in. Now you have the best of both worlds with a very minimal startup cost. That has to be better in the long run.

Hacking is not about knowing a language inside and out or even mastering the tools you use. Hacking is about solving problems. If there is an easier way to get things done, then that is the method you should use. You should always find the right tool for the job. I don't buy the argument that being a language maven helps you to pick the right language for the job. People learn how to use tools and languages together. Being an advocate of languages or tools is not black and white. There is a very large gray area where people are interested in learning both aspects of development.

With that being said, the author is developing his own language so I can understand why he is bashing the people who use IDEs. His language is new and doesn't have very much tool support. Of course he has to use emacs or whatever text editor he chooses. That doesn't mean IDEs make people suck. Tools are tools. It's how you use them.

Java One Days Two-Four Recap

2008-05-11T20:35:00.012-10:00

Day Two started off bright and early by following the hordes of Java One attendees.

Followed by Oracle's senior vice president pitching Oracle's new developer tool.

Hackers during ScriptBowl competition. It was cool because we could vote for the best scripting language after every round via sms.

Crazy Lines waiting for sessions. The event staff did their best to organize the lines, but after each session it was a mad dash to get to your designated areas.

On one side of the Moscone Center, there were gaming stations with more bean bags and chairs. They had people playing halo and wii tennis. There was also a rockband competition.

Me and Mark ending up eating at Ciopinno's down by the wharf. The garlic dungeness crab was great.

Instead of eating bagels and donuts for the third day in a row, we ended up going to Mel's Diner right by the hotel. It was expensive and mediocre. Kind of like a mainland Zippy's.

Cool blue keynote with IBM.

A slide from the Mylan session. I used mylyn a long-time ago and it seems to have been improved on a quite a bit. I intend to check it out again in the near future.

This was one of the lines coming out of the "Choosing your java web developement framework". The largest conference room was standing room only.

Cool presentation from Atlassian. Pimp my build!

Cole Hamels pitching to a Giant. Who is the giant?

Cole Hamels was in a bit of a rough spot so there was a conference with his team.

Charlie Manuel and Bruce Bochi.

The Flyin' Hawaiian, Shane Victorino. Go Hawaii!

Can't wait till next year. Worth every penny.

Java One, Day One Recap

2008-05-07T09:39:00.009-10:00

Registration! Around the hotel and the convention center Sun put up conference flags. It was kinda cool to walk around town looking for a place to eat and see the banners up. Registration was extremely easy. We entered our email address were given badges to wear around our next. They include RFID chips that they use to scan us into sesssions that we registered for and also to randomly pick us for more swag.

The cool thing is that this year they installed biometric sensors on all of the doors so they can track when people enter and exit. They brought up some telemetry streams that show when the presentation would start and end. What happens if there is a big spike of people leaving during the middle of the session?

After registration we headed to a huge conference hall with gigantic screens all over the place. We were greeted with people dancing to hip hop. Very weird. I took some videos, which will go up on you tube a little bit later.

James Gosling and his crew have a tradition of having a tshirt toss before they start talking. A girl next to us got a shirt. I hate her.

The CEO of Sun came up to talk about the mantra of JavaOne this year, which is Java + You. The motivation of the mantra is that software is no longer driven by enterprise needs. It is driven by consumers, thus a lot of the sessions this year talk about RIAs with JavaFX, web applications with J* scripting languages, and user interfaces that help people improve their lives.

Next up was a huge session load. Most of the sessions were really good. The best one by far was the More Effective Java with Josh Bloch. He highlighted new chapters in the 2nd edition of Effective Java. I picked up a copy at the JavaOne bookstore and plan on sharing a tech talk.

Sadly there was one session that was very boring with the presenters just reading off the slides and not adding any other information. Lame.

Onto the best part, the swag! Me and Mark walked around the pavilion area to get some cool gear and talk to people to see what's new out there in software land. The first booth we hit was Atlassian's, where we scored shirts and 10% off new/upgrades of their tools.

It was funny because we decided to hit each booth in order, but we ended up going towards the booths with the best swag. The first picture of the collage shows Mark with all of the gear we got. We had to make a pitstop at the hotel cause our shoulders were sore.

The lesson for our company when we go to the con in Orlando is to bring good swag. Pens and golf balls don't do it.

The first day of Java One was pretty long. I ended up getting back to the hotel around 11 after all of the sessions and eating dinner. Here is my schedule for today:

930-1030: The Script Bowl: A Rapidfire comparison of scripting languages
1050-1150: Improving application performance with Monitoring and Profiling Tools
130-230: JRuby on Rails: Web Development Evolved
250-350: Creating a compelling user interface
410-510: Putting 3D earth into your applications and webpages
630-730: How to build restful clients with javascript, ruby and java fx
830-920: The future of testing: How community engagement is changing the rules

Boys Day / Java One

2008-05-05T20:56:00.002-10:00

It's Boys Day and the day before Java One. I'm planning on putting up pics and keep a log of what's going down in frisco.

It's going to be a busy day tommorow with a mad dash for swag and loot at 11:30. My schedule tommorow is looking good:

830am - 1030am : Sun General Session: Java + You
1050am - 1150am: Fortress: A Next-Generation Programming Language Brought to You by Sun
1130am - 1210pm: The great swag dash + lunch
1210pm - 110pm: More Effective Java. Josh Bloch!
130pm - 300pm: Sun General Session Java-Centricity: Leveraging Java Technology at the hub of your Digital Life
440pm - 550pm: Real World, Real Time, Instant Results: Make Information Work for You
600pm - 700pm: SCA: Flexible and Agile Composition of Distributed Service-Oriented Architecture Applications
730pm - 820pm: Practical Applications of Static Java™ Technology-Based Bytecode Analysis and Transformation

Pics + News + Videos(maybe) to come!

We still need the Sensorbase

2008-03-30T23:16:00.008-10:00

There is a Hackystat database discussion going on that is causing some confusion. I'm not explaining myself well so I'm going to draw some pictures with the hope of clearing things up. I'm glad we are having this conversation because it is helping to flesh out the Hackystat conversations sparked at work. We all have to have courage to express new ideas.

Philip says in the thread:

It would be extremely easy, maybe one day's work, to extend your RoR application to support just the subset of the SensorBase REST API for PUTting of sensor data. Once that's done, you have no need of the sensorbase at all; you can install Hackystat sensors, point them at your RoR app, and have your own custom two-tier, database-driven, client-server web application that is compatible with Hackystat sensor data. You now have total control and can do processing whereever you see fit.

I do believe that we still will need the Sensorbase because it does a great job of collecting data and providing the basic API for retrieving data. There is nothing wrong with the way the SensorDataBrowser retrieves data from the Sensorbase.

Here is the layout of how the current architecture is layed out in my mind:

We are limited by having only one service, the Sensorbase, as the interface for all services. This reduces the flexibility of the architecture because we are stuck if we want to write an extension or optimization that doesn't fit in with the current Sensorbase. For example, we might want to build snapshots or aggregates on the database end.

Here is the architectural layout with MySQL behind the Sensorbase instead of Derby:

Having MySQL (or Derby in networked mode or any other database allowing multiple connections) adds to what clients can do with the current architecture rather than removing the need for the Sensorbase. We would have added flexibility because we can have multiple ways to get data. In the picture above, we could get the data from the current services, or from the new DPD service that we envision, or the Hackystat view can directly access the database (although the view should use the services).

In the case of the new DPD service, the same API would be used, but the data retrieval and calculation implementation will be changed. Since the same API will be used, the changes would be transparent to the clients.

For example, clients would still be able to get DPD Commit data using a webservice call regardless of which DPD implementation is used:

http://hackystat.ics.hawaii.edu/dailyprojectdata/commit/johnson@hawaii.edu/Default/2006-01-31T00:00:00.000

If another public service is built on top of the DPD service, there would be no problems because the API would be the same.

Philip also writes:

To me, going down the path of expanding the role of your forked SensorBase into doing analyses opens up a fairly large can of worms that you might live to regret down the road. As just one danger, you could end up maintaining your own, highly idiosyncratic version of the SensorBase that is incompatible with the "public" version and so you cannot integrate new analyses and tools that others are contributing to the Hackystat ecosystem.

We don't want to replace the current sensorbase, we just want to add another service on the same level as the current sensorbase. We would still be able to use all of the 'public' features provided by the offical Sensorbase and services because the APIs for the services would remain unchanged. The problem, as Philip mentions, is we would have to maintain all of the custom services and anyone who doesn't use MySQL will not be able to use our optimizations.

So in conclusion, we don't want to move towards a two-tiered, data-base driven web application. We want to utilize the benefits of the current RESTful architecture and caching, while also using the benefits of a RMDB.

Hopefully that clears up what I wrote in the email. Whether or not that is the right way to go is still to be determined. Philip also says, "It seems far better to figure out how to solve your performance issues within the current architectural boundaries, which mimic the web's in general and so are pretty much time-tested." After talking with Aaron, I don't believe that caching will be removed because there will be other 'public' services that still will be using it. I agree that we should investigate the current caching mechanism and plan on testing it against my workplace's data this week.

DailyProjectData with Rails

2008-03-25T01:42:00.007-10:00

The past month or so has been spent setting up Hackystat at work and hacking on a Rails Daily Project Data viewer. So far the going has been pretty slow. Several things came up like working on the Netbeans Innovators Grant proposal and preparing for a intern speech at the University of Hawaii. Excuses, excuses I know.

The good thing is that tonight I came across a nifty Rails charting plugin called Open Flash Chart. A cool hacker wrote a Rails wrapper around the php package. I did some quick plugin installation, copied some example code, and hooked it up to the data retrieved from the Daily Project Data webservice:

The charts on the left are embedded flash charts that I created using data from the CodeIssue and Unit Test DPDs. The charts are very responsive and have very nice eye candy. Hovering over a section in the chart will display extra data in a tool tip. If you take a look at the pie chart, I hovered over the pink section to display a tooltip saying, '94 Findbugs Issues'. Each section can also have embedded links to forward you to another page. In my case I would want to send the user to another page with a drill down of the type of data they selected.

These are the two charts I played around with tonight. I was thinking that it would be cool to display DevEvent data over the course of a day or a chart with Commit's and Issue information.

The current use case of the viewer is to create a information "dashboard" that can be used in 10- minute stand up meetings. The goal is to have the people in the meeting quickly interpret the data to drive discussions. In version 7, Daily Project Data information was displayed in a table. There was a lot of text and it was quite hard to get a feel for things without diving deeper into the data. I'm hoping the the using charts will help alleviate that issue.

I find it very cool that the new architecture allows us to have multiple viewer projects going. David and Pavel have the SensorDataBrowser and ProjectViewer modules up and running, but I really wanted to do some Rails hacking. So I just fired up a service and created my own view. I love it.

The source for the project is hosted on the Google Code Site. There isn't much there besides the source, but I wanted a place to commit my code. Josh Stupplebeen is also going to help out since he wants to get back into hacking Hackystat and wants more Rails experience.

Wants to hack Hackystat. Check.
Wants to learn Rails too. Check.

Sounds like a winner!

Code Review Lessons Learned

2008-03-18T01:41:00.002-10:00

I moderated a code review today which provided some interesting lessons learned.

Scale review scope according to code complexity. We just got through with a milestone release, so I decided that it was a good time to review our algorithm code we have been hacking on during the iteration. It turns out that the one hour allotted for code reviews was way below the time required to review the code. I am talking about 400 lines of code that takes 2+ to review for correctness. I originally was relying on lines of code as a good indicator on how much code I should request for a review. Bad idea! The reviewers felt rushed and were unable to follow the code logic because the review scope was too large. So the lesson here is to not rely on lines of code when selecting the source code to review. If the code is very complex, as it was in our case, it is probably a good idea to focus on certain aspects to limit the scope.
Provide explanations of external library method calls. I got a lot of questions about the external API method invocations, which were found everywhere in my code. Next time, I think it will be a good idea to provide brief explanations of what each method call does.
Provide the source code for external libraries. If you have performed a review with Jupiter, you know how important IDE features are when digging deep into someone's source. In lesson learned #2, I said that it is important to tell reviewers what external methods do. I think it is equally important to give reviewers the ability to jump into the method calls source to read the documentation or read the external source code. What I found during this review is that a lot of my code was skipped over because reviewers didn't know what the external method calls were doing. All they had to go on was the name and my inline comments. They ended up taking a lot of things for granted and assumed that everything was fine and dandy. That is the exact opposite behavior that I want reviewers to have. If they are going to find interesting and helpful issues, they must question everything! No one writes perfect code. Especially not me.

Code reviews makes everyone on your team better. I can't wait to try Crucible.

Think Higher

2008-03-16T01:12:00.003-10:00

My company's annual performance evaluations ended a couple of weeks ago. In the technical and teamwork areas I did fine. The big knock against me was my lack of domain knowledge. I code ok, work hard, try to get things done, but I don't have a high level picture of what my work is trying to accomplish. In other words, I do what I'm told, but I'm not as helpful when it comes down to figuring out what it is we are actually supposed to do.

That turns out to be a huge thing. Stepping up to the next level means getting to that point where you are bringing ideas to the table rather than waiting to be served. Sounds easy...right?

So my next task is to figure out how to gain some domain knowledge. How does that even happen? I can't put anything under my pillow in hopes that I'll gain some knowledge from the domain fairy. At my Referentia Internship program pitch last Thursday at UH, I told potential interns about the importance of gaining domain knowledge. I told them how gaining some domain knowledge will separate them from the rest of the pack. I really believe what I said and I'm working on it as we speak.

Over the last two weeks, I've been trying to reach that point where I'm thinking less about implementation and more about solving problems. I'm trying to reach new levels of abstractions. I'm trying to Think Higher.

Rather than wondering what type of design pattern would fit best, I'm trying to move up the thought ladder. What types of things fit into the workflow of our users? Is this useful? Where should our development focus be?

I am hoping that by changing the way I work, I'll be moving on towards gaining that elusive domain knowledge. One of our tech leads at work has been helping me out. In my opinion, his greatest strength is this higher thinking process I'm blogging about. He is mentoring me and trying to grow new problem solving approaches in me. Hooray for mentors!

So what's next? I think that the next step would be to Think Higher about Hackystat. Rather than trying to figure out how to implement my DailyProjectDetails viewer in Rails and with cool ajax forms and charts and... It's easy to get lost in the implementation details. I want to start thinking about why would the viewer even be useful? What types of views and charts and forms would make it easier to provide the information that they needed. What is the purpose of this project? Once I find that purpose, how can I improve it and make it useful in it's domain?

So add this to the list. Gain Domain Knowledge. The year is moving by quickly.

Netbeans Innovators Grant

2008-03-09T00:22:00.005-10:00

The last week and a half was spent editing a proposal for the Netbeans Innovators Grant, which awards developers with $2000 or $11,500 to work on a project that will benefit the Netbeans community. The idea that one of my team members had was to write a port of Jupiter to Netbeans. What a great idea! Our company performs code reviews and it is a huge pain to moderate the review when half of the reviewers are using pen and paper and the other half is using Jupiter.

Don't worry. I'm really am writing a blog about proposing to get a grant to work on Netbeans. I may be an Eclipse fan boy, I am also a huge proponent of using the right tool for the job. If one day Netbeans leaps over Eclipse, I'll be the first one to switch. I think that the Netbeans team did an awesome job integrating Ruby on Rails development. I am currently am writing a DailyProjectDetails viewer using rails. I put the Hackystat mini-feed on hold, while I try to pump out a useful view that we can use at work. The viewer should be appearing in another blog as soon as I get something that can be deployed.

Getting back to the Netbeans grant, I thought the whole process was enjoyable and painful at the same time. I must admit that writing proposals suck. I have helped to write a bunch at work and took the lead on the Netbeans proposal. It sucks because it is mentally draining and you spend so much time getting the flow and content just right. I literally spent hours searching online for current review tool research and solutions.

The enjoyable part is the end goal. The motivation factor. I really do believe that we can work together to create something useful. If I can spend a week living in pain to have 4 months of fun working together on a project, then I will sit there and pump out proposals all day long ;)

Working on this proposal helped me realize that proposal writing is skill that everyone should have. Originally I thought that writing proposals were not for me. Why should I write proposals? I'm a hacker, just let me hack. I'm never going to need to have this proposal writing skill. It is a good thing that I got some proposal writing practice because it turned out to be extremely useful.

"What is the current technologies that exist in your proposed area?"

"What is our project's special sauce?"

The two questions above were implanted in my brain and helped me to write the proposal. I used those two questions to help me find justification for writing the proposal. I looked for the current integrated review tool solutions and pitched our idea. I then talked about what our "special sauce" could be. I really believe that the project is do-able and provides value.

Hopefully the Netbeans grant team wants to drink our Jupiter Kool-Aid too. It tastes great.

We need a Hackystat developer config script

2008-02-24T01:16:00.006-10:00

I recently setup my machine to run Ubuntu Gutsy, so that means that I have to install all of the apps and redo my Hackystat configuration. I decided to put off reconfiguring Hackystat until I really needed to since it's takes awhile. Tonight I wanted to run the latest Sensorbase so I needed to setup my environmental variables. Here are the libraries I had to manually download and setup to create Sensorbase binary. *deep breath*

Junit 4.4
Pmd 4.1
Restlet
Derby
Findbugs 1.3.2
Javamail

Followed by creating the following environmental variables:

JUNIT_HOME=/home/austen/sideProjects/external_libs/junit-4.4
PMD_HOME=/home/austen/sideProjects/external_libs/pmd-4.1
RESTLET_HOME=/home/austen/sideProjects/external_libs/restlet-1.0.
DERBY_HOME=/home/austen/sideProjects/external_libs/db-derby-10.3.2.1-bin
FINDBUGS_HOME=/home/austen/sideProjects/external_libs/findbugs-1.3.2
JAVAMAIL_HOME=/home/austen/sideProjects/external_libs/javamail-1.4.1

Then I found out that I need to build the Hackystat Utilities module before I could create the Sensorbase binary. More downloading...

Apache Jcs 1.3
Apache Common Logging 1.1

And more configuration...

APACHE_JCS_HOME=/home/austen/sideProjects/external_libs/jcs-1.3
APACHE_COMMONS_LOGGING_HOME=/home/austen/sideProjects/external_libs/commons-logging-1.1
HACKYSTAT_UTILITIES_HOME=/home/austen/sideProjects/hackystat/hackystat-utilities

Phew. That was quite a bit of setup just to build one jar file.

Another thing that bugs me was I downloaded the wrong version of Apache Commons Logging. I downloaded the latest release, which happened to be v1.1.1 instead of v1.1. Since I have done the setup before I automatically knew what was wrong, but new developers might not get it right away. I'm thinking that the error messages should specify what jar file the build wants.

What we need is to have a little shell script that goes out and grabs the current files, extracts them to the specified location, and sets your environmental variables up. So all those steps turns into two.

Write the appropriate install folder location.
Invoke the script from the shell of your choice.

Any takers?

Back to Ubuntu

2008-02-19T23:37:00.002-10:00

Ubuntu. Dual Monitor Display. Happiness.

I'm sitting here writing this blog anxiously waiting for my laptop to finish defragging so I can shrink my windows partition with gParted. I tried to partition it early without defragging, but I got an error so I'm sitting here doing nothing while its takes >2 hours to complete.

I used to run an earlier Ubuntu distro on my desktop (Breezy I believe) and it was an awesome. Hacking on linux is absolutely one step towards being productive and getting things done. Linux sometimes pisses me off since some of my hardware doesn't work, but every new release of Ubuntu fixes something that wasn't compatible. I remember the days when the only solution to get my wireless adapter to work was to write my own drivers. Hacking + Linux is great. Here is why I think you should switch.

Fast file system. ext3 > NTFS. This is an absolute must-have when compiling. I remember way back in Hackystat 6 and 7 where compiling on linux would be up to 5 minutes faster than windows.
Multiple desktop support. Let's face it. Windows has crappy multiple desktop support. I tried Microsoft Power Tools, a free app that let's you have multiple desktops. Crap. Slow. Not configurable. Aaron informs me that he has had multiple desktop support with his other computer. Sadly it was tied to his ATI video card. No dice.

As a hacker you need lots of screen real-estate. So much that you would need gigantic monitors to fit everything you want. Multiple desktops let you have Eclipse open, multiple consoles, a text editor, browsers, instant messengers, twitter, your mail client, downloads, all open at the same time. You set up your environment so you know exactly what desktop screen the app you lives on. You move fast. You have to or else the IM blinking won't stop.
A useful shell. To make you hacking life easier, you need to be able to grep the output of your app, pipe some information to a file, or just be able to copy and paste without using the stupid "mark" option in the Windows shell.

Linux is awesome. When it works. Some people complain about their hardware not working or just not having the same amount of software choices. I feel their pain. Linux is not for the weak. It grinds you until you want to just give up. Then you find this post about someone who wrote a script that fixes some obscure file that was causing your install script to fail. Hooray. Some people don't want to deal with that. I had enormous amounts of trouble setting up dual monitors in previous releases. That and my special Kensington Expert Mouse doesn't work. I too sometimes that Linux would just work.

If you know how much of a pain setting up dual monitors was, you'll be happy to know that setting it up with Gutsy is supposedly easier. No more xorg.conf hacking. So I have heard. 54% completed. Compacting files. 46% more until I can leave you windows.

So long and thanks for all the fish.

Mini-Feed Webservice Making a Comeback?

2008-01-31T00:42:00.000-10:00

The Hackystat Mini-Feed has hit a road block. A Mini-Feed webservice may be the answer.

What I want to do

I had the idea that the mini-feeds should display some aggregated information based on batches of the sensor data entries. This will allow the feed to say things like:

Austen currently has 25 minutes of active time in the past hour. 2 hours total for today.

@ 12:02pm Austen has invoked an ant build.

@ 4:04pm Philip has run the Emma Coverage tool. The foo module currently has 99.9% method level coverage.

@2:02am Aaron has commited 25 files.

The Problem

The issue I had tonight was the large amount of webservice requests required to get the right data. There are basically two ways to request project data in Hackystat. [1] Request the index of sensor data entries, which provides you with information such as the URL to the entries' representation, a timestamp, sdt, user, etc. [2] Use the URL found in the index to drill down into the entities' representation.

To implement the Mini-Feed, I need to drill down into the entities' representation to get the entries' runtime and data type specific stuff.

For those of you wondering what the difference between timestamps and runtime are, timestamps are unique to all data entries. It acts as the 'primary key' of an entry. A runtime can exist in multiple entries and provides a way to 'batch' data together. An example of a batch of data would be all of the entries associated with an unit test tool's execution. Each batch would have information about the unit test's execution, such as whether or not a test passed or failed.

Drilling down into each data entry is time-consuming due to the large number of data entries. If the index has 5000 entries, I will have to make 5000 separate requests to get the entry specific information. My tests of performing 3000 or so web requests to my local sensorbase took about 45 seconds to complete. Thats just too long. I think the overhead of making all those requests is causing the slowdown. Thats my theory anyways...

The solution?

So I was thinking "How come the DailyProjectData services don't have that problem too?" I believe the reason is that there is only one webservice call binding each entries' data to Java objects.

So in order to remove the large amount of requests I will have to create a Mini-Feed service that can talk directly to the SensorBaseClient. I am wondering if caching will get rid of this problem. If so, I probably should hold off on writing a Mini-Feed service.

Hackystat Mini-Feed Part 2

2008-01-23T00:58:00.000-10:00

This weekend I did a bit of ruby hacking to parse out the sensor data information from the Sensorbase webservice calls and found it to be relatively easy. Ruby is so nice ;) As I was hacking I had this feeling that I was placing the logic to parse the data in the wrong place. For my quick prototype, bunching all of the logic together was fine. But it turns out I need a higher level of abstraction in the form of a new service. Philip explains in more detail to Pavel and Dan Port in his email about creating a new service.

In order to separate the view from the mini-feed business logic, I need a new service that can allow clients to retrieve the abstracted mini-feed information.

This new service will allow multiple views such as the Rails webapp I'm writing, RSS feed viewers (Maybe RSS isn't a view, but I want people to have that information available via RSS), twitter, and other apps to be able to request the mini-feed information via webservice calls.

Now what type of information can be requested?

I've only gotten through half of the RESTful Webservices book so I only know how (relatively speaking since I'm still a REST noob) to design a service allowing clients to GET information. I'm probably going to take a look at the telemetry service or the SensorBaseClient class in order to figure out how to get my service working.

With the new service comes a Mini-Feed REST API that I must design. I have to think of what type of information can be requested by clients. Here are some of my initial thoughts:

- Mini-feed information spanning all projects, kind of like what twitter does on it's public timeline.
- Relevant mini-feed information for a specific project.
- Relevant mini-feed information for a set of users
- The start and end time of the information can be specified to reduce/expand the grain size.
- Webservice calls to filter the data. (Explanation below)

The ideas I just listed above isn't anything new. You can get all of that information using the sensorbase REST API. I'm thinking that abstracting the webservice calls to a higher level will help filter the data that isn't useful for mini-feed purposes. Clients lose the flexibility of getting all the data, but will hopefully gain an easier and smaller interface to get the information they need.

So what is this filtering I'm talking about? Pavel and Aaron turned me onto Facebook's News Feed preferences, which allows you to customize the information that appears in your News Feed. The cool thing is that it doesn't completely remove all of the information that you don't like. It just shows it less frequently.

The goal would be to get this type of control in my Rails webapp and request the user's customized information using the "Mini-Feed service data filter" REST API.

How does that sound?

Hackystat Stream of Development

2008-01-17T23:52:00.000-10:00

Tonight I started working on the Hackystat version of Atlassian's Stream of Development Consciousness, which will tell mini-stories about what is currently happening in your development shop. Much like Facebook's Mini Feed, we want to provide awareness to people about what you did, when you did it.

The first thing I did was figure out how to request Hackystat data. For that information I headed over to the Hackystat REST API Specification wiki page and took a look at what was available to me. After a bit of head scratching and help from some fellow Hackystat hackers, I finally found the URI that should provide me with some interesting information.

Now that I got past that hurdle, now I can start to think about getting my web application (written in Rails yay!) to parse and display the right information. There are still a bit of things to think about.

What type of information do we display?
I was talking to Aaron online tonight and he came up with the idea of just displaying the amount of events that have taken place. For example, one mini-stream could say, '3 hours total dev time, 50 unit test invocations with 4 failing tests, and 200 builds in the last 2 hours'.

Of course that may be enough information for our little data stream, but it would be interesting to get more context behind the collected data. Perhaps we could start add comments to the mini-feeds? What if we added some telemetry to the mini-feeds? That would be cool to see how the mini-feeds turn into a large information feed over time with a telemetry chart.

What type of information should be collected/filtered?
Currently commit data from Subversion is collected once a day. I think it would be important to see commit information in the mini-feed. Committing code is a good indicator of progress so it would be nice to get more up-to-date information with respect to commits.

Another metric that we commit once a day is Issue data from JIRA. It would be good if we could collect that data more often so that we could update the mini-feed with JIRA event information.

I think that Filemetric and Code Issue data wouldn't be as useful as UnitTest and Commit information. I don't really know about this one. Maybe someone would want to see what kinds of FindBugs errors you have ;)

Cool New Features
DrillDown information would be cool because if you find an interesting mini-event, you could click on the event link and see what types of unit test failures are happening or what classes were commited.

An RSS feed with the mini-feed information would fit nicely into my Google Desktop app.

Commenting on feeds would also be cool.

Mini-feed comparisons at the grain size of a day? Is your mini-feed activity the same during the week the same?

Imply information from the data in the mini-feed. For example, if Aaron and I commit changes to the same module or are running similar unit tests, can we say that we are working together? 'Aaron and Austen have been working on module mini-feed for 2 hours'. Totally awesome.

So fun!

Lists are fun!

2008-01-15T21:49:00.000-10:00

I'm a bit late, but I've been reading lists of what people want to accomplish this year and I think it's time to write one of my own. I personally don't believe in New Year's resolutions since I know I won't keep them, but I've done pretty well with blogging twice a week. I'm going to see if I can keep up the technical improvements this year.

Improve my writing skills
Since August I've tried to up keep this blog with technical topics and personal rants. Not all of them have been my finest work (actually a lot of them were horrible), but after making time for this blog, I think it's a great personal improvement project. You get to share ideas with the people that read your blog, work on your writing skills, and best of all, you can jot down your ideas and thoughts. I often have ideas that sound excellent in my head, but I found that things aren't so clear once you start to talk about it. I notice that I come up with these great one sentence ideas for blogs and have trouble writing 3 paragraphs. Writing it down will help my thinking process.
Ruby/Apache Wicket and Hackystat
This past year I spent time writing a Ping Pong ranking application with help from Aaron. Learning a different language was great and I now can talk about design patterns and technical topics by comparing other languages to Ruby.

Back when Hackystat 8 was being just started to be re-written, there was talk of a Ruby interface. We have decided to explore other options using Ruby or Apache Wicket. The current interface was written in GWT by David and Pavel, but I think there was some difficulty developing with GWT. Feel free to talk to them for all the nitty gritty details. Now that we have some new Hackystat developers coming into the lab, we are going to explore a new interface. I want to help out development to become part of the Ruby community ;D
Practice Puzzles
One of the themes of this year’s learning will be focused on buffing up not only my technical abilities, but to work on my thinking power. I will be the first to admit that I am not the swiftest person out there. I have trouble understanding what people are saying and piecing things together. I have a hard time visualizing concepts in my head. That is the main reason I love dry erase markers and white boards.

I am thinking I need to start giving my brain more practice. I’ve decided that I’m going to massage my brain by working on technical puzzle books. I just bought Puzzles for Programmers and Pros from Amazon and plan to work through it once I finish reading RESTful Webservices. Not only will this help me work on my brain power, it will be great practice for intern and my own interviews. We also have a large Rubik’s cube contingent at work. I plan on buying a cube to practice my visualizing skills at home.
Play with Python
I don’t really have much to say here. Python is a dynamic language that I’ve some interest in for a while. I plan on working with a lot of Ruby this year, so I probably won’t have much time for Python. Getting introduced to it this year is definitely doable. At the very least it will satisfy my dynamic language curiosities.
Buff up my outward and critical thinking
As I mentioned earlier, thinking is not one of my strong points. Aaron often talks about how he looks for interns that have outward thinking. I am not 100% sure what he means by outward thinking, but I assume he means that he wants people to think of problems that need to be solved. People should not only solve their own problems, but think outside of their domain and take a look at other problems to solve. He wants people to think critically about things that might spark some interest from others.

I totally have a deficiency in this area. I am an inward thinker meaning I usually think about implementation level details rather than the big picture. I think critically about code, process, and software development, but I don’t look at why I’m so concerned about it all. I really need to take a look at the problem first before diving in the solution.

I’m not sure how I’m going to tackle this problem this year. Hopefully people can help me along the way. At the very least, I am aware of the problem, so that is a good start.
Give a speech to a group about something in technology
I was scheduled to answer questions from Sandy Ho at the HTDC Science and Technology Fair about what Referentia is looking for and can give to it’s potential employees. Sadly the Q/A session was canceled and we ended up emailing the answers to her. This year I want to give a speech to a group about what companies are looking for in their budding software engineers. It would be nice to give a speech to students at UH and talk about the importance of reading, hacking, internships, and all the good stuff about software development. Right now I feel that UH has too much emphasis on theory and not enough on software development (aside from Philip’s class). It is up to Hawaii's hitech companies to take some initiative and provide internships that give students the experience necessary for them to succeed. Hopefully I can give a convincing speech to some people and get them motivated.

Phew. 50 weeks to go.

Edit. Add one more to the list.:

aaron: okay.
aaron: back to goal.
aaron: want you to find a hackystat research question and hypothesis.
aaron: thats your goal.

Sharing is addicting

2008-01-13T23:14:00.000-10:00

Today as I was reading some blogs on my mobile and sharing stuff, I felt that twinge. That twinge that subconsciously tells me, "Hey why are you sharing that? Everyone knows that. People are going to see that you shared that and be like WTF."

That feeling that overcame me was totally awesome. It's that feeling that makes people actually care about the quality of their software. People don't want to let their team down. Similarly, I don't want to let you all down by sharing crappy articles. After all, I know that you are all like me and are too busy coding, reading, thinking, blogging, and all that other stuff (eating, sleeping, face-to-face interaction) to waste time reading boring articles.

Thinking about it more, sharing is kind of like console gaming. You start up your Nintendo Wii, play some Wii Tennis, and dominate. You are a pro. Your rank is as high as it possibly can be. You destroy the computer. Then you go to your company summer party and get pwned by your coworker. (True story)

Sharing items lets you get out of the sandbox. The range of topics you learn about grows out of control. You might just find something that interests you. Sharing has totally infected me. Every time I login to Google Reader I try to select interesting articles that might spark some interest. Sharing items is like an implicit way of blogging. They look at the items I share on Google Reader and wonder if any of the links are worth clicking. Now I have to make sure that I read blogs often, constantly sharing new things or else people might stop reading my shared items. What will they think of me? Sharing is important, but so is the stuff that is being shared.

GeoTIFF Hell

2008-01-06T22:35:00.000-10:00

This weekend I spent a lot of time manipulating GeoTIFF images for work. GeoTIFF images have embedded meta-tags that contain coordinate, scale, and other miscellaneous information about the wrapping image. My only problem with GeoTIFFs is that the tag information is hard to access. The metadata isn't human readable so I had to use another application to read it. Man that sucks.

I had a problem where I had different pixel coordinate spaces for the GeoTIFF and the GeoTIFF's canvas. Figuring out a common way to relate the two pixel spaces was a pain because I didn't know what information was available to me. It would have been so much easier if I was able to take the graphic, look at it's metadata in a text editor (Textpad yah!), and see what I could do. What I had to do instead, was load the image using my image manipulation API and see what data was available via the debugger in Eclipse. Oh I also had to read the GeoTiff specification (yuck). What a pain. Not only did I not know what the metadata meant, I had to do this every time I wanted to see what information was available. Lots of overhead compared to reading the information in Textpad whenever I wanted. Why not just have a key-value pair of human-readable information?

Scale= 10 meters per pixel
Lat=10.10
Lon=-86
etc etc.

Aaron brought up the idea at work about images with context. He seems to enjoy looking at pictures instead of reading (blasphemy!). If there was a reliable way to search for images based on what was happening in the image rather than whatever Google does with their image search, we would be better off. This idea could be spread to all types of images. If all images had a standard text format that could be read, parsed, and manipulated, we might be able to create some interesting applications. Flicker currently allows context to be associated with images by users manually adding tags to their images. Why not embed the information in the image just like GeoTiffs? Hopefully the metadata would be of the human-readable variety.

Why Everyone Should Get Interviewed

2007-12-21T00:00:00.000-10:00

Over the past couple weeks, I was getting interviewed by Aaron. This was the first time I have been interviewed outside of a job setting. After going through the process, I think that interviews are a good way of thinking about where you are in your career, what you need to improve on, and what you should strive for. The interview process is a lot like revisiting your resume every six months(which I still have not gotten around to doing). Here are some observations I had about getting interviewed:

Having extra time to gather your thoughts for an answer is a great exercise. Every question that Aaron asked required me to stop and think about an answer that expressed how I feel. It also made re-evaluate my reasoning behind my answers. I was happy to find that I couldn't find answers to some of his questions because I now have a concrete starting point to think about where I want to go.
The interview questions should be focused on a certain area of be effective. In Aaron's interview, he asked me how I got started, what I like about developing software now, and where I want to do later. I liked how he made me think about several aspects in my career. If he didn't ask me the right questions, I wouldn't be thinking about the right things.
Using back-and-forth emails to interview a person is better than I thought. I originally thought it would be better to just send one email with a whole bunch of questions. After doing this interview, where I spent long periods of time just thinking of an answer, I was glad that each email only had one question. It felt like I was accomplishing something and I didn't half-ass the end portion of the interview.
Self-introspection is fun! You get to learn about yourself. It's interesting when you learn new things about someone you should know the best.

I definitely recommend getting interviewed. It was a great experience that I want to have again in the near future. It is actually very easy to get interviewed. Just ask a friend, neighbor, colleague to think of themselves at a bar having a few drinks with you. After everyone is feeling good, fire away with the questions. Its a relaxed atmosphere with absolutely no pressure. All upside, no downside. Anyone want to get interviewed by me?

Alcohol + Hacking != Increase in Creativity/Debugging

2007-12-17T23:19:00.000-10:00

Tonight I tried a little experiment that involved a martini and some code. I have read and heard about the glorious effects of alcohol on the ability to think differently while hacking. Alcohol supposedly can provide you with some "liquid creativity". Much like the "liquid confidence" we are all familiar with, especially at those late night karaoke parties. Thinking outside the box can be yours when you grab some booze and start hacking. All ready to be ultra productive, I had some of my drink and started coding. Here is what I thought would happen:

"Hey I see what this code is doing! Let's try something new. Why not. What else do I have to lose. Code code code hack hack hack. This is great! I should drink and code at work!"

Here is what really happened:

"Oh god I just deleted all of the class again. CTRL + Z. Grrrr why can't I type correctly. Why doesn't this work???? I totally wrote a unit test and now its failing. I didn't type anything. CTRL + Z AGAIN. What was that class called again? Search search search fix typo fix typo backspace backspeace aldkasjfasdlfjlasdfsf"

This was after half a martini. I'm a lightweight drinker, but this was ridiculous. My motor skills were not kicking in tonight when I was trying to code. Even when I had to figure out why something was working, I felt no difference in my ability to debug. I actually got more frustrated cause it didn't make sense. My thinking was just clouded from the alcohol.

Worst hacking experience. ever.

(You Hackystat hackers out there know I tried doing this the day before our we get ready for the public release. It's ok. Unit tests pass. Woohoo!)

Unit Tests Show Me The Way

2007-12-13T23:26:00.000-10:00

After a week long hiatus, I finally got back into Hackystat hacking. I re-started my work on the Commit Daily Project Data Analysis Service. After some initial startup lag (getting a drink, listening to some easy hacking music radio, etc) I was ready to whip out some code and get it done. Or so I thought. I had no idea where to begin. I only had 3 classes, but I still was lost. What was I working on a week ago? Does everything work or do I need to debug? Oh man this is going to be a long night...

Then I remembered that I wrote unit tests as I was coding. I did a quick ALT+SHIFT+X+T and saw the little green bar appear in eclipse. (I'm running unit tests with a keyboard shortcut for those of you who don't use eclipse) Hooray, it seems that everything I did in that class is functional. I did this for the remaining two classes and was able to figure out relatively quickly what I completed the previous week.

Writing unit tests while coding continues to amaze me with it's usefulness. I can add "restarting long forgotten work" to the existing list of unit test uses such as "debugging while I code" and "ensuring the correctness of my logic". I remember long ago that someone told me that unit tests help developers learn about the intent of someone's code. I guess this is a prime example of showing someone (me!) what the hell I did last week.

JAXB Pet-Peeve

2007-12-09T21:50:00.000-10:00

A while back Randy Cox made a comment on his blog about his teammate disliking JAXB:

"Another aspect of our differences is Brian's drive to build a high quality system from the start, while I tend to like to start with the basics and increase scope incrementally. Brian's strong dislike of JaxB, resulted in a lot of work on the Model to mitigate the JaxB limitations."

Curious about what the limitations of JAXB were, I made a comment on his blog. Sadly I was left out to dry on the matter and forgot about it for a month. This past week at work I revisited my JAXB code and figured out one JAXB limitation.

There is no easy to abstract JAXB auto-generated classes.

So what normally happens with JAXB is:

Create an XML schema
Run the XJC compiler, which creates the JAXB auto-generated classes.
Write the java code using the JAXB classes to marshall and unmarshall data.

The problem occurs when you have 2 JAXB objects that do similar things:

Apple Object Interface:
setName()
setColor()

Orange Object Interface:
setName()
setColor()

So now you have 2 classes that perform the same operations. You end having to write code specific to each object:

Apple apple = new Apple();
apple.setName("Ringo");
apple.setColor("Red");

Orange orange = new Orange();
orange.setName("Japanese name for an orange");
orange.setColor("Orange");

This is an excellent time to abstract their behaviors to enforce the DRY principle. Why don't we go ahead and create a 'Fruit' interface that abstracts the behavior of each object? Technically you can, but because the Apple and Orange classes are auto-generated, it isn't such a good idea to edit them directly. If you ever decide to change the XML schema, you will have to re-run the XJC compiler, which in turn will overwrite your changes. Another alternative is to wrap the JAXB objects in a wrapper class to provide a common interface. If the schema changes, the auto-generated class changes will be transparent to the wrapper's clients.

Although it is possible to provide common interfaces between the JAXB objects, there is no clean way to do it. Maybe there should be a way to specify behaviors of objects to the XJC compiler. That would remove the round-about way of abstracting the JAXB classes.

I'm sure that there even more limitations to the JAXB package, but currently it's benefits outweigh the drawbacks. XML parsing is fun again! (Well, more fun than manipulating the XML DOM tree directly)

My Resume Objective Is Crap

2007-12-06T22:34:00.000-10:00

Your resume, like your objective, should give me a sense of you and where you're going. I want to see a little ego and I want to see your character because I'm not hiring a flat piece of paper, I'm hiring a person.
- Rands in Repose, A Brief Glimpse

After reading Rands' article on how to write a good resume objective, I decided to take a look at my old resume. What I found is a crappy objective statement:

To bring my knowledge and experience from research on software engineering processes to the real world; to develop software for a world class software development company.

Wow. So I want to bring it to the real world huh? And a world class software development company aye? Would it make sense to put, "to develop software for a decent company." No wonder employer's eyes glaze over when they hit the first part of potential candidate’s resumes.

After some brainstorming, I worked out a preliminary objective:

I need to work in an environment that values developers over products. When this happens I can write software that is useful and will make a difference. As a software engineer, I work hard to write quality software.

This objective says that I want to work somewhere that doesn't impose crazy deadlines and micro-management to get software out the door. It says that I want to be someplace that gives software engineers interesting problems and turns them loose. If you want quality software, give me a problem and get out of my way. This statement definitely needs work, but I think it's a good start. I think it fits the situation of "sitting at a bar, having a couple drinks, talking about software with a friend."

I also took a look at the rest of my old resume. I think it's time to wipe the dust off and update it (Stay tuned for an update in another blog). I need to know where I am now, and where I want to go.

No direction + No Awareness = Career Death.

Serve your customers

2007-12-03T23:41:00.000-10:00

The ability to accept feedback from *anybody*. I find the best feedback comes in the "customer is always right" sense, where the intern is a "customer" of your design. If they feel like something is more complicated than it ought to be, there's a reasonable chance they are right, even though their identification of the "root problem" may be laughably wrong. And even if they are completely wrong and it really does have to be that complex, a humble designer should not immediately resort to "Because I said so", but should at least try to explain why.

- http://www.codinghorror.com/blog/archives/000051.html

With a new member on my team at work, I gave him a task to implement a new feature for an upcoming milestone. He was to use code that I wrote to implement a new feature. After spending some time learning how the code works, he came up to me and said, “Austen, isn’t there an easier way to do this?” After explaining to him the reason why things are they way they are, he trodded off and churn out more code. Once again he came back to me a bit later saying, “I’ve been using your API and here are the areas that I am having a problem with.” I wondered what was wrong. I had no problems using my own code. Once again I worked through the code with him and explained it in even more detailed. A bit frustrated, he went off to hack. We ended up talking about the code a short while later.

Looking back, I learned something valuable. I learned that I should be humble, rather than handling the situation by saying:

Stop bugging me. Use my API. It’s awesome.

After the second time he came confused, I should’ve stopped and said:

Hrm. I see you are back again. Is my code too complicated to be used by others? Let’s talk about why you are having problems.

The new member on my team because a “customer” of my design. He brought to light very large flaws in my original design. My code ended up being a lot more complex that it should have been. This was evident in the large amount of crazy complicated code required to implement the upcoming feature.

I believe that you don’t write code for yourself, you write it for others. After all, you should know the in’s and outs of your code since you wrote it. If you belonged to a one-person team, cowboyed it up, and wrote the next Mac OS, then you don't need to be clear about what you are doing. But if you are like the rest of the software world, your code will be read by your team. If you hide the intent of your code or don’t document what’s expected, your team will be confused. It's great that you know if you pass in a value of ‘42’ to your method, you find out the answer to the meaning of life. But if you don’t make it obvious, someone who has never read The Hitchhikers Guide to the Galaxy isn’t going to know what the hell you are talking about. If someone tells you “Hey your API sucks.” Maybe it does suck and you should stop and investigate. They are doing you a huge favor by helping you pin point the areas of suckiness. You should aim to please your customers. After all, they are always right.

On a side note, don’t be afraid to suck. Suck less every year.

Random Recursion Thoughts

2007-11-26T00:23:00.000-10:00

I was thinking about recursion earlier today so I decided to do some snooping around. Wikipedia has a nice section on recursion. Back in my college days, I learned a nifty little language called Scheme. Some of my classmates totally hated that language. They had trouble getting around the huge amounts of nested parentheses and the endless horde of recursion. Recursion here, recursion there, car, cdr, cdr, car. (Don't worry I also forgot what car and cdr mean in Scheme) In my opinion, recursion is awesome. I don't use it very often in practice, but I learned how to think about function calls by learning recursion.

I heard some one say that that Scheme is a lot like Lisp. I know that the authors of Xkcd think that it is a godly language.

After researching a bit, I did some simple recursion problems. Reversing characters, computing numbers, etc. It made me miss recursion...

I wonder what students are learning today. Has recursion been phased out?

Do you know me Hackystat?

2007-11-19T00:00:00.000-10:00

James turned me on to a streaming music website called Pandora. Great website. Good music. Nice eye candy. If you haven't been to Pandora before, I will break it down for you. They allow you to give a thumbs up or thumbs down to currently streaming song in the genre of your choice. From there they do some "music selection magic" and select a different song that usually is something you'll like. They probably use similar ratings from other users with the same musical tastes. So far I'm really happy with Pandora. I'm hearing bands I've never heard of before that I'm starting to like. But the most interesting part of Pandora is the feeling I get when I'm logged on. It's uncanny. I feel like Pandora knows me. With music selections, it feels like it knows who I am.

With all the social networking, open social, and collective intelligence talk floating around the internet, I have been thinking that it would be cool if Hackystat would go further and try to "understand the people behind the metrics." We currently collect project data and developer data to a certain extent. From there we infer certain things like the health and direction of project or a developer's activities. What we don't do is analyze the people responsible for the direction of the project. There are many negative issues surrounding the use of developer metrics to analyze people. A big thing is that there is no way to accurately represent who a developer is with the data that is collected. There are just too many intagibles behind software development. Using developer level metrics in a scientifically proven way is hard and we have not solved that problem yet. But what if we did?

Would more people embrace Hackystat if there was a way for people to relate to the their own data? Maybe their is some type of hidden disconnect between the metrics and people. I know that I personally don't log onto Hackystat and say, "Wow this tool really reflects who I am as a developer." Is that really useful? I don't know, but wouldn't it be awesome if you could feel the same type of emotional connection just like when Pandora starts playing another awesome song (which it did just now).

Developer level metrics are interesting. I hope we can find an interesting and useful way to utilize it.

Reviewing Code Reviews

2007-11-16T00:33:00.000-10:00

This is an additional post to Aaron's blog about a review I moderated.

Over the past couple of months, I've tried to increase my code reviewing soft skill. Early on I gathered feedback about my ability to moderate code reviews. Like any noob, I had some glaring holes in my approach. I'll try to summarize my faults so you can recognize them and hopefully avoid the mistakes I have made.

No offense and defense - The first couple of reviews I attended/moderated, I attacked peoples code and defended issues I found. I learned that it is very important not to think about reviews in terms of being on the offensive when pointing out issues and playing "tough defense" when explaining why issue x should be solve with that new awesome design pattern you found in Head First Design Patterns. You must be respectful of everyone's code. More often than not, they spent 100x longer deciding whether to make a certain decision. If something is weird, point it out, and ask for their input in a respectful manner.
Give people the benefit of the doubt - People are smart. Most times they are smarter than you. (At least it seems that way. Maybe I'm just slow...) It's silly to think that you can read someone's code for an hour and immediately know the tradeoffs of the current design. People know what they are doing and usually do things with purpose. This leads directly to my next point.
Stop, Shut up, and Listen - Listen carefully to what people have to say. Don't dismiss the explanations about why they chose to do something. There may be a specific reason for their choice. Again it just not productive to think that you know everything about their code.
Provide guidelines, not rules - So you went ahead and read Effective Java. Twice. You know the chapters by heart and know the horrors of using inheritance instead of composition. You start reviewing someone's code, and *gasp*. Inheritance. Inheritance everywhere with no composition in sight. You take out that red pen and start slashing away marking up every place that uses inheritance. During the review you cry, "Don't use inheritance or else" except the only problem is that no one is listening to you. If you want people to listen to you, provide suggestions or alternatives. You must remember that every software engineering principle and rule is not set in stone. In the end only the developer will decide what needs to be changed. If they aren't listening to you, they won't change their code.
Ask others for their opinion - Once again, people are smart. You will be amazed about how people think about problems. It is important to ask others what they think about the current issue. I've done this many times and have found new ways to look at code. A simple, "Is that the right way to do it?" or "How would you suggest we fix this code?" will do the trick.

Code reviews are great opportunity to have some fun and learn new things. Everyone is there to improve the quality of the code base and increase the team's hacking fitness. I'm planning on attending and moderating a lot more code reviews in the future. It is a learning process, but I think that I am moving forward. Hopefully I am changing out of the "know-it-all hot shot hacker that is set in stone about his methods" mold to the "nice, approachable developer that will take the time to help me without biting off my head for asking questions" type of hacker.

Blogging, Open-Source Software, and Rockstars!

2007-11-11T21:49:00.000-10:00

Software by Rob posted a link titled: 100 Resources to Attract, Retain, and Use Rockstar Programmers. One-third of the links are job boards, but below those are some interesting articles about how people in the industry choose their developers. I read two articles that caught my eye, The Engineering Value of Blogging and Hire Programmers From Open Source.

I've talked to some developers that do not see the value of blogging, but I do feel that being able to write your software thoughts, troubles, and experiences down really helps me get a clear idea of what I'm doing. You can think of blogging as doing your software design on paper. You don't just sit in front of your workstation and start whipping out code. You slow down, grab some paper, and draw out how you are going to solve problems x, y, and z. When you blog, it helps you stop and think about what has happened. You can reflect on a interesting blog you found or bring up technical problems in your current open-source project.

Both blogging and open source software are techniques to improve yourself as a developer. Well-known software bloggers often write about making yourself marketable because you never know when you will need to look for another job. Who knows what could happen. Maybe your employer is running through tough times or you just don't like what you are doing. Making yourself marketable doesn't happen overnight. It will take many years of hardwork, discipline and motivation. As a noob developer, I'm still starting on this process. It is tough staying disciplined, but I know that it will be worth it.

Give people all the information about what type of software developer you are. Help interviewers out. They are the ones that need to figure out if you are right for their company. Blogging and Open Source contributions give them that information. Type your name into Google. I know your interviewer will.

FogBugz Evidence Based Scheduling Part 1

2007-11-09T01:25:00.000-10:00

The other night I was sweeping through my feed reader as usual when I came across a blog talking about FogBugz Evidence Based Scheduler.

In evidence-based scheduling a completion date is constructed from developer estimates, but the completion date is actually a range of dates with probabilities that each date will be met. So you can look at the calendar and say “We have a 50% chance of finishing by October 31st, and a 90% chance of finishing by November 23rd.” In addition, the schedule is based on the developers’ past estimating accuracy, so the better their estimates (compared to their actual time spent), the tighter the completion timeframe. Bad estimators means a wider timeframe.

"Wow", I thought. That sounds pretty good since I am a horrible estimator. I can't give a good estimate until I get waist deep into my task. I ask Aaron about it and he shuts me down:

Austen: [Evidence Based Scheduling] is awesome.
Aaron: you think Evidence Based Scheduling is awesome?
Aaron: Write a blog on how that sucks.
Aaron: estimates probably sucks. I think software engineering research people would eat that up and show how it sucks.
Austen: I think its good. Why do u think estimates suck? is it because they are never accurate?
Aaron: First of all, it is leap. Leap sucks. It is almost like PSP. Psp sucks. Read Philip's and Ann Diseny's papers on that.

So my next task is to check out Philip and Ann Disney's Papers on PSP Data Quality. Hopefully it will help me understand why estimation "sucks."

Meticulosity!

2007-11-05T01:05:00.000-10:00

Meticulous : marked by extreme or excessive care in the consideration or treatment of details.

I have found out this week that one of my new goals is to be meticulous. This week alone I have made three stupid mistakes that could have been avoided if I pay a little more attention to detail. Here are the stupid mistakes:

Totally messed up merging my subversion sensor with the main line of ant sensors. Take a look how I overwrote all of Julie's and Aaron's hard work. Philip found out that there was lots of white spaces in the build sensor. Some inspection showed that all of the ant sensors were reverted back to before my branching. Yikes.
After doing some work build hacking, I broke the build because I didn't test all of the modules. I tested one module, saw that it passed, and went to sleep. I wake up at 7 to find out every other module failed. Bleh.
Tonight, I spent an hour or so trying to figure out why SCLC would not work. With Julie's awesome help I found out that I was using ActivePerl 5.6 rather than 5.8. sigh

Being a bit more meticulous should lead to less headaches. I think this is the reason that I love unit testing. When I'm hacking I write some code, write some tests, and forget about it. I don't have to worry about my code being correct because I've tested to see if it works as I have intended. If I do find a bug, I do some debugging to figure out the cause, then write a test case to reproduce the problem. I can only image what types of horrors I would bring out if I didn't have unit testing.

Now all I need is some way to test my process.

Assert.fail("Don't be a noob you are overwriting everyone's hard work!")
Assert.fail("You didn't try every case that might cause the build to fail.")
Assert.fail("Wrong PERL version!")

Or maybe I should stop hacking at 12am...

Sometimes I hate Ant

2007-11-02T00:53:00.001-10:00

I wonder what the Ant designers had in mind when they decided to not include the ability to have if-else logic. Tonight I started work on some Ant macros to simplify the invoking of Hackystat version 8 sensors on the current set of Hackystat 8 projects. I wanted the ability to print out a message saying, "You dont have the Hackystat 8 ant sensors installed so I'm not sending data." Seems pretty straight forward.

Check if the ant sensors exist.
If the ant sensor doesn't exist, print out a message, do not send data.
If the ant sensor does exist, send the data, drink some beer.

What that simple if-else logic mutates into is:

Write a task that checks for the antsensor jar file.
Write a task that invokes the sensor if the jar file exists.
Write a task that prints a message if the sensor doesn't exist.
Write a task that invokes the three previous tasks.

I spare you the 50 or so lines of ant code that I need to write just to have some simple if-else logic. I find it incredibly bogus that if-else can only be represented by setting properties and including an "if or unless" attribute in the target tasks. Not having a target embedded conditional task just makes me angry. The Ant-Contrib Ant task extension attempts to add additional features that Ant doesn't include.

Can't I just have a built in if-else clause?

On a side note, I noticed that version 7 sensor data sending code would fail the build if the sensor was not installed. Hackystat developers really should be "eating their own dog food", so it probably will be OK to require that they install their sensors.

Skill vs. Will

2007-10-28T22:41:00.000-10:00

If you haven't noticed yet, I am very big on self-improvement. I am currently reading a book called Managing Humans: Biting and Humorous Tales of a Software Engineering Manager by Michael Lopp, who also writes the blog Rands in Repose. While reading this interesting and witty book, I came across a section called Skill vs. Will. It explains four states of developers:

High skill, low will: Boredom is imminent-needs a change of scenery and responsibility stat.
High will, low skill: Needs training, needs mentorship. Needs management. The good news is they really, really want it. Savor this because as soon as the skill kicks in, they're going to starting your job. This rules.
Low will, low skill: Boy, did you screw up. It takes a fairly concerted effort to ignore the needs of your employee so long that (a) they no longer have the skills necessary to do their job, and (b) they don't want to do it. Roll those sleeves up, pal. You've got work to do.
High skill, high will: Great job, ummmm, guess what? No one stays here long.

There are several things that came to mind when I read this section.

Why does management screw up if you have low will or low skill? Isn't it the responsibility of the developer to ensure that they keep up on the latest development styles, techniques, and processes?
Having a great motivation to learn is a great state of mind to have. You may not be the best at what you do or know everything, but being motivated gets you half way there. Improving everyday can only help your chances at becoming awesome.
He seems to side-step the problem of having low will by saying that by increasing a person's skill, they will also see an increase in their will. Somehow I don't really buy that. If you already have low will, what will make you want to increase your skills? And the larger problem is that if you have low will, what is going to stop you from going back to that state once someone isn't prodding you into improving?

On a side note: Aaron starred a blog from Coding Horror about achieving blogging success. That article has inspired me to stick to a blogging plan. I'm committing myself to blog twice a week, Sundays and Thursdays. Let's see if I can stick to it for a year.

Hacking Fitness

2007-10-14T23:46:00.000-10:00

I have read numerous articles in Muscle and Fitness that explain why body building is so tough. Body builders often talk about difficulties in being meticulous in what they eat, how they train, and their supplement cycles. The most important point they bring up is that body building isn't a job, it's a lifestyle.

In my opinion, hacking should be the same way. Hacking isn't just something you do at work to pay the bills. You need to always be improving your "Hacking Fitness". I believe that in order to become an awesome hacker, you need to change the way you live. You need to constantly train your mind to think in code, refine your hacking skills, and keep improving. English (or your native tongue) should be your second language.

Another thing to improve your Hacking Fitness is to find the right motivation. You have to _want_ to improve. If hacking is your hobby, then you'll be hacking at home. If you are hacking at home you are already have a leg up on a lot of developers. You have effectively turned "work" into "play", which to me is the real key of improving your Hacking Fitness.

Some other miscellaneous items that I am doing to try improve my Hacking Fitness:

Read books. You must read. There is no way that you can learn effectively without bombarding your brain with the stuff found in all of the reading material out there. There is absolutely no excuse not to read blogs, books, magazines. If you are awesome and not reading, let me know how you do it.
Read other peoples code. You can learn bunches of things when you read another developer's code. Improving your code reading skills will increase the speed at which you figure out things. Just imagine what it would be like writing a book report when you had trouble reading the book. It's hard to make connections when there is a language barrier. The same applies to reading code.
Pair program and Pair review. I have found that learning how other people think about code and their code tracing process is very useful. While pair programming is not useful all the time (although some people swear by pair programming), I find that after you experience it with another developer, you start to think like them when you are coding. I noticed this a while back when I started to trace through code like my coworker. You start wrapping the code around your head in different ways. You see the little things that you would not have seen before. Totally useful.
Try new things. I try not to get stuck using the same tools all the time. I try to move over to something new to see what is out there. For example, I was using the Netbeans IDE for a bit. You can't find anything better if you aren't looking.

A lot of these things have been said before. You need to keep learning, you need to get motivated, blah blah blah. If you take anything away from this blog, it should be that:

Hacking isn't a job, it's a lifestyle.

Liskov Substitution and Simple Factories

2007-10-03T20:12:00.001-10:00

I came across a nice article on Liskov Substitution that clearly explained what it means to follow the principle. For those you who don't wish to read up on Liskov's Substitution Principle, the basic idea is that any sub-class should work, without any code change, for any interface that accepts a parent type. This got me thinking about the way I'm currently implementing the Simple Factory Pattern. The factory pattern provides you the ability of, "Insulating the creation of objects from their usage. This allows for new derived types to be introduced with no change to the code that uses the base class." So the implementation in Java would be:

public class SimpleFactory {
/** Prevents direct instantation. */
private SimpleFactory(){
}

public static FooObject getInstance(BaseParameter baseParameter) {
if(baseObject instanceof FirstChildParameter){
// return FirstFooObject sub-class instance using first child parameters.
}
else if(baseObject instanceof SecondChildParameter){
// return SecondFooObject sub-class instance using second child parameters.
}
// etc, etc.
}
}

This implementation violates the Liskov Subsitution Principle, which is found in reputable books such as The Elements of Java Style, because each new sub-type of BaseParameter that is created would require a code change to the simple factory. Perhaps this is one those cases where there is a trade off between code brittleness and encapsulation. While I agree that it is nice to encapsulate how objects are created using a factory because the method calling code does not have to worry about what type of object to create, I find the code smell behind my factory implementation to be disturbing. First, large numbers of subclasses would have to rely on type-specific code that constructs and returns the appropriate instances. Second, if a library with the SimpleFactory is in use by other developers, they are hosed if they create their own BaseParameter subclass. The code will not be able to be changed because Factory methods are static, therefore unable to be sub-classed.

The Liskov Substitution Principle is something I'm going to have to think about when designing new code. It is nice to finally get a solid grasp on what it means to violate this well-known principle. Perhaps my implementation of the Simple Factory is incorrect and there is an implementation that does not violate the principle. I should read more code.

I'm still learning

2007-09-27T23:14:00.000-10:00

Not only are changes in objective inevitable, changes in development strategy and technique are also inevitable. The throw-one-away concept is itself just an acceptance of the fact that as one learns, he changes the design.

I'm currently plowing through the Mythical Man Month by Fred Brooks and came across the above quote tonight. I find it funny and interesting how things he writes apply to software development today. Man Month was written over 30 years ago, yet we see the beginning of agile ideals. Change is going to happen, so we might as well be prepared for it.

Fast forward to today and I find that I'm learning and changing everyday as I'm developing. A good example is my littering of static factory methods in my code. A year ago I read Effective Java and said to myself, "Hey these static factory methods are awesome. Having names for your "construction" methods would be crazy good!" What really ended up happening was:

private FooObject(Parameters){
// Do stuff with parameters
}

public static FooObject createFooObject(Parameters){
return FooObject(Parameters);
}

I did that for almost every object. I can see the benefits of being able to return any type of object, but all that leads to is unnecessary casting of objects. There are certain places where static factory methods are useful, but I didn't really gain anything except having more code to maintain and document. What a headache.

So now I have learned and I have changed my design. Every thing I read, learn, and practice is being tucked away into my hacker toolbox. While it may be frustrating to have to re-design my code, it is a great feeling to be able to look at a problem and think of multiple ways to approach it. Someone said that if all you have is a hammer, you'll only see nails. I want the whole hardware store.

IDE's should not have to compete on their suite of available plugins

2007-09-21T00:13:00.000-10:00

The title of this blog is inspired by a one of my very smart coworkers, "IDEs should not have to compete on their suite of available plugins, but on their core editor's usability." My coworker said this in reference to Jupiter, which is an Eclipse review tool plugin. My coworker is a Netbeans users and was asked to use Jupiter to speed up the review process. At work there is a collossal IDE war. In one camp we have the Netbeans user's who feel that introducing features that existed 3 years ago in Eclipse is the way to go. And in the other camp we have the superior Eclipse users. (Can you tell I am a happy Eclipse user?) I am over exaggerating, both IDEs have their pros and cons. But the point is that users shouldn't have to chose an IDE based on what suite of plugins are available. Netbeans users should be able to use Jupiter. Developers shouldn't have to learn the Netbeans API in order to write a Jupiter plugin with the same core functionality.

What my coworker suggested was the creation of a Generic Framework that allowed other developers to create "IDE Adapters", which would allow a plugin to be written only once. For example, in the case of Jupiter, a developer would be able to write a Jupiter plugin against the Generic IDE Framework. The plugin could then be adapted to the developers IDE of choice. Sounds so simple doesn't it? One plugin that can be ported to multiple IDEs.

The tradeoff is that the Generic API will lose the IDE functionality that is not common to all IDEs as my coworker says: "The API would be limited to the lowest common denominator of features." How much functionality will be lost? Who knows, maybe too much will be lost because certain features do not exist in others.

In general I think this is a interesting, complex, useful idea. If we could get everyone using the same set of tools regardless of IDE preference, we would be a lot more productive. Maybe there wouldn't be an IDE war. There would be peace in HackerLand with Netbeans users wondering what it would be like to code ultra fast with all of the sweet Eclipse shortcuts and Eclipse users thinking, "Maybe a Swing based IDE isn't so bad after all".

A dream perhaps?

I think we need Hackystaller Version 2.0

2007-09-17T19:25:00.000-10:00

Last night I was preparing for the September 17, 2007 milestone release of XmlData. Since the last milestone produced problems with the 'ant -f verify.build.xml' invocation and with a missing dependency check of Hackystat_Utilities, I decided to attempt to build from source on my desktop. My desktop had no Hackystat services or Java code analysis tools. I spent about 45 minutes getting my environment able to run a 'ant -f verify.build.xml'. I found that I had to download and configure the environmental variables of all of the tools found at the Hackystat-Sensorbase-UH Developer Guide.

XmlData requires Sensorshell and Hackystat utilities, which in turn requires Sensorbase. I hope I never have to format my machine because this takes quite a while. Hackystat 7 was much easier because all of dependencies (If i remember correctly) were stored in your ant library directory.

I'm thinking that eventually we will be needing a tool like HackyInstaller, but something better. Let's call this tool HackyDevInstaller. It probably should be able to:

Allow the user to choose the source and distribution releases of their Hackystat service(s) of choice.
Prompt the user to download the modules that the selected services depend on.
Automatically download the right analysis tools, install them, and allow you to configure the environmental variables.
It should also manage the user's current installation so they can remove or change their current settings.
After developing some Ruby on Rails, I found that using Instant Rails was great for quickly starting development. It would be great if we could start a contained Hackystat development environment that started a local sensorbase service and provided some debugging utilities.

Perhaps we can find a NSF Research Grant for Undergraduates for a boy and girl who love hacking.

I'm pushing back reading once again.

2007-09-14T00:18:00.000-10:00

So rather than read a few extra pages of Steve McConnell's Professional Software Development, I'm writing a blog. It's so hard these days to find the time to read in-between work hacking, Hackystat hacking, and my new man-crush, blogging. Aaron pointed me to a new book that I immediately ordered from Amazon. It's called Software Creativity 2.0 and it looks very promising. Robert Glass has written some very good books including the "F-book". Why is it called the "F-book?" Perhaps you need to read it or you can see the obvious and figure it out for yourself.

So the creativity book by Robert Glass contains a forward by Tom DeMarco the co-author of Peopleware. (I'm not going to link it because the 2 people that actually read this blog know the awesomness known as Peopleware). Top that off with a amazon.com review by Steve McConnell and you have one hell of a book to look forward to.

With my spontaneous purchase of Software Creativity, I began thinking about what I should be reading. In my amazon wishlist, I have a bunch of books in a range of topics. These topics include Java, Ruby, design patterns, agile practices, interface design, and general motivation and creativity books. I normally try to buy 2 books from amazon at a time for the free shipping and to vary what I read. Normally I get a tech book and generic "how to become awesome" book, but I always wonder if I should be focusing on certain things. Maybe I should be focusing on improving my Java 1 billion % or perhaps I should get all the Ruby books I can because it just might be the next big thing. It is pretty hard to read everything nowdays, so I really have to be selective. I wonder how all you big readers / hackers / workers / parents get it done. I think I'm going to sleep now, but what a waste. I could be reading or hacking or blogging more.

How do we petition to increase the amount of hours in a day to 48?

XmlData: The results are in

2007-08-31T23:58:00.000-10:00

After running three test runs of the XmlData sensor, I have gotten the sensor up and running. The three test run settings were:

The default JVM heap size, which is 64ish MB.
An increase of JVM heap size to 1024 MB with a manual invocation of System.gc() after each send to the sensorbase and after each file.
An increase of JVM heap size to 1024 MB.

After I fixed the problem of loading the hashmap with duplicate entries, the sensor failed to finish sending data due to an Out of Memory Exception. After some investigation, I found that the sensor was failing when the largest file was unmarshalled. The largest file was 10 MB in size.

First Test Run with no increase in heap:

So what I decided to do was increase the heap size and found that the sensor completes. Great success! At 1:13, you can see the largest spike on the heap usage graph. That is the time when the 10 MB file was unmarshalled. With the increase in heap, the sensor had no problems.

For fun, I ran one last invocation of the sensor with out the manual System.gc() calls. It turns out that the sensor allocates around 26 more MB to send the same amount of data. It is interesting to note that at the end of the sensor process, the data is very small, but the sensor still uses the same amount of allocated memory. In the 2nd invocation, the heap usage graph shows that it only uses a minimal amount of memory.

I will need to document in the user guide to increase the heap size if sending large files. It might be a good idea to tell users to increase their heap to a large number. If we come across the case where a user has an abnormally large file, we may need to go the route that Philip suggested of writing a custom SAX parser. That will remove the need to load the entire document into memory.

The sensor is almost functional. I noticed that version 7 data includes pMap attributes, which I probably will need to separate. After thats completed, I can write some documentation and release! Hopefully I can send out a review request at the next milestone.

Monitoring Java Processes with JConsole

2007-08-28T23:39:00.000-10:00

I am currently working on fixing some Out of Memory (OOM) exceptions that are being thrown by the Hackystat XmlData sensor. Philip suggested in an email that the a good first step would be to start looking at heap usage with JConsole. JConsole is actually a decent tool. It gives you graphs of heap usage over time, spawned threads, created classes, and a nice little over view. The tool even lets you invoke the garbage collector on the monitored process.

The Overview of the monitored process.

The Heap Monitoring Tab

Tonight I was trying to reproduce the OOM exception, which from the exception log, was thrown when a 10MB file was unmarshalled. I tried to reproduce the error by only sending data from the 10MB file, but the exception was not thrown. After consulting the screenshots I took of the previous night's test run, I found out that OOM exception was thrown after an hour of sending data. I also noticed that the heap usage hovered around the high 50MB to low 60MB range.

My hypothesis is that the memory usage was almost at a maximum when the large 10MB file was accessed. So if I can keep the heap usage leveled off, maybe the exception will not get thrown. To test my hypothesis, I pressed on the "Perform GC" button a couple times while sending data. I noticed that the heap usage would stay at the mid to high 50MB range. That led me to insert some 'System.gc()' calls after each file and after sending a batch of data. The test run is currently running and I do notice that the heap usage is about 10MB lower on average.

Wow it looks like you get a blog with updates before the blog is posted! The exception was just thrown again, but the heap was around 49MB when it encounter the 10MB file. I'm beginning to think that the solution may be to increase the JVM heap from the command-line. I did a test invocation with a 40 MB data file and the OOM exception was thrown immediately. If the sensor sends all of the data with 1024MB of heap allocated, we may just have to inform users to increase the heap size.

Requesting the user to increase the heap size in this case may be the correct solution. Previously, increasing the heap did not work because I implemented the sensor wrong. I need to write a blog entitled, "It's usually your fault." We'll just have to wait and see how the current test run goes.

Hackystat 8 Supercharged!

2007-08-23T00:44:00.000-10:00

As I was in the shower 5 minutes ago, I had a thought. (I think I should move my workstation into the shower because I channel the most ideas there).

The Hackystat 8 architecture will work. The reason why I believe it will work it is a combination of old and new things:

Disciplined Developers
Great Hackers
The new motto: "Convention over Configuration".
The sweet new architecture known as Version 8.

Disciplined Developers
Pavel found a great article of what it takes to be a disciplined developer. Without discipline I don't think you will have the ability to produce quality software in a timely manner. If you look up discipline in the dictionary, you'll come to this page.

Great Hackers
If there is one thing I miss about working in CSDL, it's reading other people's code. Working on Hackystat 8 has reminded me what it is like to read beautiful, elegant code. A motto I have come to accept is to "strive to be the worst in the group". The only way to improve is to always have something to learn from your peers. The Hackystat team is constantly providing me with that opportunity ;)

Convention over Configuration
The motto "Convention over Configuration" is a large improvement over the Hackystat 7 motto, which was, "Config it, then let's document!" (There wasn't really a motto for Version 7, so I decided to make one up for entertainment value). In Hackystat 7, you had to go through so much application and dependency configuration via properties files. That was a huge headache if you start developing on another machine.

In Hackystat 8, things are already starting to look very different. Gone are the days of endless configuration! We now have small little modules, each with their own library that can be used by other modules. Want to start the server up to receive data? Configuration files? No way! Hit the command-line (java -jar sensorbase.jar). I'm liking it so far. Small little pieces, small little hacks, large contributions to the Hackystat project.

Sweet New Architecture
I'm not going to pretend I can explain the new architecture in detail. Check out Philip's Version 8 Design Doc. Sweet!

As Aaron has said, Hackystat is fun. I can't wait till the shirts come out. I'll take 10.

ant and c++ bad times...bad times.

2007-05-09T00:13:00.000-10:00

I'm beginning to think that ant and C++ just don't mix. After hours of struggling with getting the ant-contrib cc task to link object files and static libraries to create an executable, I've just about given up on trying to get this to work. There have been two big issues that have been bugging me with ant-contrib:

g++ gets called when compiling the source files, but gcc is called when linking. This is a huge issue for me. When I call g++ from the command-line to compile and link my source and object files, I have no problems. Everything works great. After using ant's verbose mode, I noticed that even if you specify 'g++' as the linker to use, the gcc linker is still called. This leads to unreferenced files and more C++ hell.

I did some searching around and noticed people having the same problems, but no work around.
The library is buggy. There is a bug where ant will hang when nesting 'syslibset' and 'libset' tags within the cc, compiler, and linker tasks. The work around is to have nested calls within your cc task. You get no love if you have a lot of libraries to link.

A coworker suggested that I look into Nant as an alternative to using ant. I'm not sure if I really want to use Nant. The whole point of using ant to build java and C++ projects is to have one build system. I know that I can call Nant from ant, but that will just spawn a new process, which will hide the error results.

I guess it's time to re-learn makefiles.