The Simple Fix That Took Way Too Long

As Unity 2018.4 LTS is reaching EOL now I was tasked to upgrade the Unity version of the game I'm working on. The upgrade was quite easy and after a couple of days everything seemed to work. When doing the final test before we were supposed to ship the game built with this version however, I found a problem when changing profile picture. It opened the gallery app fine and allowed me to pick a photo, but as soon as I did the game crashed! 

The crashlog pointed me to the open source cropping library used by the Image And Video Picker asset we are using in this project, saying something about not finding the class. Both ClassNotFoundException and NoClassDefFoundError were shown, so I assumed that for some reason the JAR file wasn't included in the build anymore. It was also searching in some really weird paths, so I thought it could have something to do with obfuscation as well. I started going through all config files, gradle scripts and android manifests that had been changed during the upgrade, but couldn't really find anything relevant. I did a few changes and built new versions only to find the problem remaining after every try. I took to google and thought surely someone else must have had this problem, but I couldn't really find anything.

I then decided to upgrade the asset from the Unity Asset Store as I found there was a new version during my googling, but still the same error remained. At this point I finally looked closer at the error and realised it said:

Failed resolution of: Leu/janmuller/android/simplecropimage/R$layout;          eu.janmuller.android.simplecropimage.CropImage.onCreate (CropImage.java:111)

I went to the open source repo and found that line, and it was when trying to load the layout it failed:

setContentView(R.layout.cropimage);

This made me realise I'd been chasing the wrong problem. Apparently the CropImage class had been loaded, which is in the JAR, but it failed when loading the layout from resources. I unzipped an old APK and a new APK and sure enough the resources from the library was present in the old one that worked, but not in the new. 

This made me remember that old warning I've seen in Unity about putting android resources in res folders being deprecated, so I started reading up on that. Apparently what you need to do is to either put your resources in an AAR or in something called an "Android Library", that is basically an exploded AAR where you have a folder with a project.properties and it's own manifest, and a special folder structure inside that contains the assets. The CropLib however seemed to be a proper "Android Library", so I couldn't understand why it didn't work. I went googling more and still couldn't find any good resources to solve this, so I decided that once I find the solution to this problem I must publish it on my blog to make sure no one else has to spend all this time on this.

I went for lunch and an appointment with the barber, and when I came back I opened up the documentation for using Android Libraries again with restored energy, and now I noticed that Google had given me the documentation for Unity 2019.4. I changed the little drop-down in the upper left corner to 2020.3 and there I finally found the solution! In the end of the article it said:

Android Library projects must have the .androidlib extension for Unity to support them. Add this extension to your library’s root folder name (for example, mylibrary.androidlib), and place the folder in the Assets folder of your Unity Project.

This is apparently new in Unity 2020, changed by the rational that you now don't have to put it in the Plugins folder, and the asset had not been updated for this in Asset Store. So, I renamed CropLib to CropLib.androidlib, built a new version, and it worked! So in the end, I spent about two full days on something that turned out to be solved by just renaming a folder! One could have thought that the Unity upgrade should have done this, or at least pointed it out, but at least I didn't see it do that.

If anyone else have this problem and finds this blog post useful, please make a comment to make me feel better about spending all that time figuring this out.

My Way: Working With Versions

Today I want to share with you my favourite way of working with versions. I've been around, so to speak, and seen different ways, and this is the way I would suggest.

First thing we do is to create a branch called "develop", off which almost all work is done.

• Tiny fixes that are just one commit, I usually do directly on the "develop" branch.
• Small tasks, that are usually completed within a day, and with a single contributor, I suggest branching off from "develop" into a kebab-case-name in the folder "dev/*".
• Bigger tasks, that might go on for several days and with more than one person contributing, I suggest branching off from "develop" into the folder "feature/*". What's important is to never have more than one of these long living branches, and that you often (at least daily) merge develop into the feature branch.

When the time come for us to do a release, we create a branch in the folder "release/*" called major.minor.x based on "develop". If we're about to release version 1.39 for example, the branch would be called "release/1.39.x". The 'x' is because the number of iterations that will be needed to stabilize the release is unknown at this point. 

Once we have the branch we change the version number in the project to 1.39.0 and make an initial build. Immediately following that we update the version number to 1.39.1, so we know that and logs, tracking or bug reports saying 1.39.0 is that exact build that we just did. During the time between builds we have the odd revision number up until the point when we're doing the next build for testing. Then we update version to 1.39.2, make the build, and immediately followed by updating to 1.39.3. Get the drill?

Any work being done during the release process that isn't meant to stabilize the release should still be done in "develop", following to the bullet list above. It is also good practice to continuously merge the release branch back to "develop" to make sure that the issues are solved also there. Just make sure that you don't merge it back when it is an even revision number!

Finally, when we deem the product ready to meet the world, we merge the release branch to "master", making "master" always be the latest released version of the project. We also merge the release branch back to develop one final time where it is integrated with all work that might have been done there while stabilizing the release.

I hope this article can help some people to improve their release process! I'm also curious to learn if any of you have any practices you think are better or if you have any concerns with my proposed way of working. Let me know in the comments!

See also: 

• How to Commit Code

Farewell to an Amazing Job

I have an amazing job. I work as a game developer in the new games studio for one of the leading mobile games companies in the world, experimenting with different game ideas to find the next big hit. While that is awesome, the company also earns an insane amount of money, so I can pretty much get whatever I want, as long as I can motivate it. Could it be better? I don't know, it sure seems pretty good when I put it that way. But still, that apparently isn't enough, because I just resigned. Am I crazy? Yeah, probably. Anyway, as some people have asked, here's my reasoning.

First of all, I've been here for more than five years by now, and as this is my first job in the games industry I'm kind of curious to learn how other companies operating in the same area do it. The new company I'm going to work for is also making mobile games, but they do somewhat different games and they do it in a completely different way. While the game engine is the same as I've been using the last few years, the team size and composition is very different. And I'm really curious to learn what else is different.

Second, while being this successful and having all the money in the world is kind of convenient, it also makes it very hard to really make an impact. Making a new game or a feature that earns a million dollars in a company valued to more than five billion dollars is something else than doing the same in a company valued to 35 million dollars. Also in a less monetary sense, you're a much bigger part of the company when there is less than a hundred employees rather than over two thousand. I'm hoping that will also make the company more personal, with less policies and processes that can't be bent and more space for doing the smart choices.

And third, it come down to the actual games in the making. The company I am joining have a strategy to focus on social multiplayer games, something that is very dear to me. While my current company also have an ambition to do that, it just doesn't seem to really happen. To be fair though, it is more easily done with the games of the new company, where you actually play with others, compared to the primarily single player games we make here and add a social meta layer on.

So, to sum it up, I'm leaving a great place with great people, to learn more, to have a greater opportunity to make a difference, and to make games I can be even more passionate about. I really would like to thank everyone here for this amazing time, and for all the faith you've put in me. It's been a blast!

Reboot Complete

Last I posted here was five years ago when I published My Reboot Manifesto.
Geez, what happened?

I guess it would be fair to sum it up real quick with these words;
WOW! WHAT A RIDE!

I have not regretted for a single second that I took that leap. I learned so much and met so many great people. I've made an impact on so many millions of peoples lives and helped make tons of money to both the company and to charity, and a little bit to myself too. And it totally drowned me in swag! It has sure been the ride of my life!

But it is coming to an end, and with that I thought it would be interesting to revisit the "manifesto". So let me just walk through it bullet by bullet, and please comment if you don't agree so we can talk it out. =)

Working with products for my employer instead of consulting for external clients

Round one is definitely for the better, for me I might add. I really enjoyed working for my own company's good rather than helping others, and with the added benefit of no time reporting it wasn't even a close game.

Using a MacBook Pro with OS X instead of a Dell with Windows

Another win! I generally don't really like Apple to be honest, but the MacBook Pro is an outstanding product, both the hardware and software. I really don't want to go back to Windows ever again.

Programming in Java with IntelliJ instead of C# with Visual Studio

This is a closer one. I didn't really become friends with Java, but I did like IntelliJ. However, the way it went down I only stayed with Java for about a year, after that I changed position and did C++ for a couple of years - where I used AppCode (also from JetBrains that do IntelliJ), and after that I actually came back to C#, but now using Rider (also from JetBrains) as editor. And I must admit C# is the better of these languages, but as the JetBrains products are superior to Visual Studio I'm gonna call this a tie.

Writing tests with JUnit and Mockachino instead of NUnit and FakeItEasy

This one was for the worse. While JUnit and NUnit is kind of the same, Mockachino had nothing on FakeItEasy. Also I do like the testing syntax in C# better than in Java, it's even more for the worse. Luckily that sorted itself as mentioned in the previous section.

Persisting data with Hibernate to MySQL instead of Entity Framework to MS SQL

To be honest, I can't really remember this in enough detail to judge it, as it was only during the first year I did any persistence using database. After that I've been working with mobile client as platform, where I've either persisted on the device as files or by rpc calls to a server where the actual persisting was made by our backend developers.

Having SCM in Mercurial with SourceTree instead of Subversion with TortoiseSVN

Here's another tough one. Mercurial was quite confusing to me at first, but I eventually got to know it pretty good - just in time before we decided to change to Git. After that Git was quite confusing to me, but now I've gotten to know it pretty good too... but it sure took some time. Subversion was so much easier, and the time spent on helping less technical team mates was a lot less. And in our case I'm not really sure if the benefits of the distributed SCM was actually something we benefit from. Now when I'm comfortable with Git I kind of like it, especially I like the partial commits and the rebasing, but it does come to a cost. Think I'll just call this a tie, as it depends on the situation and skill levels.

Doing CI with Jenkins and Ant instead of CruiseControl and Make

I'm afraid I have to give this to CruiceControl, mainly due to all the problems we have had with Jenkins here. I really like Jenkins when it works, especially the Jenkins Pipeline, but CruiseControl just worked, and that's actually the top most feature I want in a build server.

Handling sprints and issues with Jira instead of Redmine

What to say about this one... I guess I don't really like either of them. Well, actually I guess I don't really like bug-tracking-systems at all. I think that Jira has the upper hand on Redmine though, as it's so much more widely used, and it's so customisable and have so many plugins, so it is hard to be a competitor.

My Reboot Manifesto

So, I wanted to learn something new.
An opportunity appeared and I changed employer.
And with that, everything changed!

Working with products for my employer instead of consulting for external clients
Using a MacBook Pro with OS X instead of a Dell with Windows
Programming in Java with IntelliJ instead of C# with Visual Studio
Writing tests with JUnit and Mockachino instead of NUnit and FakeItEasy
Persisting data with Hibernate to MySQL instead of Entity Framework to MS SQL
Having SCM in Mercurial with SourceTree instead of Subversion with TortoiseSVN
Doing CI with Jenkins and Ant instead of CruiseControl and Make
Handling sprints and issues with Jira instead of Redmine

That is, while there is great value in the items
on the right, the items on the left are new to me.

I'm in for a real deep dive, and I love it!

Now I learn a lot every day!

My concern about Story Points proved right!

This months PragPub Magazine contains a great article by Ron Jeffries called "Estimation is Evil". It contains a couple of great quotes I'd like to share, and also proves my concern about Story Points right.

In my previous post "Story Points vs Hours" about a year ago I wrote:

"The more unreasonable reason is that as estimates are hard to do, you put a metric on them that is hard to understand to get away with them easier. If you say that something will take three days it's easy to see if you were right, while estimating it will take three Story Points will keep you safe. Who are to say how long a Story Point is? Hardcore agilists laughs at such a silly question."

In Ron's article he writes:

"There are a number of ideas about how to estimate using something other than time. Points, Gummi Bears, Fibonacci numbers, T-shirt sizes. These were originally invented to obscure the time aspect, so that management wouldn’t be tempted to misuse the estimates. (I know: I was there when they were invented. I may actually have invented Points. If I did, I’m sorry now.)"

Wow! Reading that just forced me to write this post!

However, since I wrote my previous post I've had the chance to use Story Points and was pretty happy doing it. We started with the assumption of a story point being half a day. That way, when estimating, we could easily transform our estimates, and thinking about "half days" instead of hours made the estimations simpler. "Would you finish this is half a day? One day? Two days?" is more tangible and easier to answer than doing the same with hours.

The story point estimates keeps being translated to hours on another level though, and my most frequent question is what formula to use for that transformation now. That's alright with my though, I can stick with estimating in half days and calling it Story Points. I'm just not sure about the formula, is half a day 3 hours, 3.5 hours or 4 hours? We started with the assumption of it being 4 hours, but as we usually are faster than we estimate with that formula I'm about to lower it...

Well, back to the article... Here are some other great quotes from it:

"Most of us were taught to write down all our requirements at the very beginning of the project. There are only three things wrong with this: “requirements,” “the very beginning,” and “all.” At the very beginning, we know less about our project than we’ll ever know again."

"Anyone who has ever looked at a list of “requirements” has seen some items that were very important, and some that were—well—not so important. Not so important like 1/100th as important as the most important things. Not so important like downright bad ideas. There is a very strong “80-20” rule at work in requirements lists. The bulk of the value comes from a very small subset of the so-called requirements. So these other things aren’t “requirements” at all. They’re ideas, and some of them are not very good."

"It seems that “they” often want to know how long something is going to take, and how much it will cost. My view is that “they” don’t even know what they want, so we bloody well can’t possibly know how long it will take. However, “they” are often powerful and have the money we need, so we need to answer their question, even though we cannot."

It's a really long article, but well worth reading. You will find the full article here:
Estimation is Evil (PragPub)

What is Agile?

So most companies now say they "use Agile". It's the new black! Every one does it. Or at least says they do... So, what is Agile?

In February 2001 seventeen people met to talk and try to find common ground. Representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming, and others sympathetic to the need for an alternative to documentation driven, heavyweight software development processes convened. What emerged was the Agile Manifesto.

So, Agile is what the Agile Manifesto values. It is also the twelve principles behind the manifesto. Nothing more.

Mistaken for being Agile

There are a lot of practices that Agile practitioners use, which are really good practices indeed, but doing a few of them does not make you Agile.

• Daily stand up/Daily scrum is not Agile, it's a meeting to coordinate todays work.
• Scrum board/Kanban board is not Agile, it's a board to track what's being done.
• Continuous integration is not Agile, it is a server that builds your project.
• Pair programming is not Agile, it is two people working closely.
• Automated tests are not Agile, but they are really helpful!
• TDD is not Agile, it is a practice for good design.

The list goes on and on. None of these are mentioned in the manifesto or the principles, but they all fit well while doing Agile.

Requirements for being Agile

While the practices above are good, and should be used in many cases, they don't make you Agile. Being Agile is following the manifesto values and the principles behind it. If you don't, here's some news: You're not Agile!

Here's a little list of interpretations I did from the manifesto and the principles.

You're not agile if...

• Delivery of working software isn't done on at least a monthly basis
• The progress isn't shown to the stakeholders on at least a weekly basis
• You're not willing to change the requirements along the way
• Business people and Developers don't talk to each other
• Face-to-face conversations among team members are rare
• The team or the stakeholders work a lot of overtime
• You don't start simple, doing only the most important things first
• Reflection on how you work, and adjustments accordingly, don't happen regularly

Conclusion

Please stop saying you do Agile Development based on having a daily standup meeting and a board with your stories divided into tasks. That is not what being Agile is. It would fit well in a Waterfall project too. Adding sprints and retrospectives takes you a little closer, but you're still not there. To be Agile you need to be able to check the list above on how you're not Agile... and if you're on the list, you're not Agile.

That being said, I don't want you to stop doing the practices you're doing. They are good practices, which will help you deliver better software. Just stop saying you're doing Agile unless you really do.

Beware the Agile Smells

For a while now I've been somewhat disturbed by all the methodologies I've so eagerly studied during the last few years. You know, the Agile movement and the Craftsmanship movement that's been coming on strongly. I've been troubled by all the time that is consumed with other activities than coding in my projects.

When I first saw the Programming Motherf**ker manifesto I thought it was ridiculous, but I'm not all that inveighs now. Even though I'd still call it silly it has some kind of point, because lately I've consciously been moving away from safety and coming closer to performing. I'm backing a couple of steps toward where I once was, the pragmatic programmer moving quickly just doing stuff. That was before I was infected by Agile.

Slowing down

Doing agile is a lot about safety where you time box stuff in short iterations with up front estimates, which is good in many ways. But when you spend too much time preparing, planning and estimating it stabs you in the back in the end. Your velocity would be better if you just did something, instead of four of us playing planning poker about if this is a one or two point story, and how you'd best divide it in tasks. I've also had problems with endless retrospective meetings and ad hoc discussions about how we work.

Also TDD and Unit Testing can have the same problem; it can easily be overdone. It's great in so many ways, but you need to keep a good balance between safety and productivity. I'm a big fan of Unit Testing, don't get me wrong, it's just that I don't think you need to test everything. Nor do I think you need to inject every single dependency. More on that later.

Speeding up

I came to notice a course by Dan North called Faster Software Delivery, held in Norway, which sounded really interesting to me. It seems to address exactly this! But since I live in Sweden I asked him on Twitter if there was any book or screencast on the subject, but unfortunately no. A book is in the writing though! However he pointed me to a talk he's been doing on a couple of conferences called Patterns of Effective Delivery. I found a good webcast of it here. You really should watch it when you're done reading this post!

Facebook Poster

Another interesting talk I've found lately is this 10 minute speech from JavaZone by Christin Gorman. She complains about the way of coding taught by Uncle Bob where everything should be broken down in methods of ideally one line of code, and naming that method properly so it's obvious what it does, to make the code more readable. I'm not quite sure who's side I'm on, but I find the discussion really interesting, and the best way is probably somewhere between the extremes.

At Facebook they have a culture inherited from Mark Zuckerberg encouraging to do stuff fast. Mark has this saying, "Move fast and break things", claiming that if things don't break you're not moving fast enough. Just shortly ago we often saw that in production, but now it seems they've either slowed down or put a wall of testers between the fast moving programmers and the production environment as I haven't had much problems with Facebook lately. They also have it written on their walls that "Done is better than perfect" to remind themselves to always keep shipping... that's something I really can relate too and actually put up that poster on a wall next to my desk!

Agile Smells

Talking too much

Since I started doing Agile I've had a lot of more meetings than before. Meeting are good though, making sure we're doing the right thing and keeping every stakeholder up to date with the outcome. But since we have them frequently it's important to keep them short and to have them when needed.

Changing too much

Being open to change is a cornerstone in Agile, and we should be – both in what we do and how we do it. But if we’re too open to change it becomes a problem. It may be adopting a new hot technique without the need for it, merely because there’s a lot of fuzz around it. It may also be not giving the ideas from the previous retrospective a fair chance before changing again or changing back to normal as we can’t manage the work to apply the decided change.

Continuity is needed for any methodology to work, so we can’t expect something to solve the problem in the first week. If something is obviously bad you should of course drop that practice directly but make sure you give your ideas a fair chance before ditching them and trying the next.

Prototyping too much

Prototyping is generally a good practice, where we invest as little as possible to gain as much knowledge as possible. We need to make sure though that we don't invest too much in the prototype though, as it would take away the benefit. Prototypes should be quick to make and shown to the stakeholders as soon as possible, and once the general idea is decided it should be terminated. Optimal efficiency is reached when the prototype is hand drawn with the client present and participating.

In some cases it may also be advisable to use tracer bullets instead of prototypes, a technique from the book The Pragmatic Programmer. Instead of creating a prototype you start writing the code with a first decent idea of how it should be and forge it to perfection through continuous feedback as you go.

Estimating too much

Estimates come to a price, and that price is that it takes time. Hence, by estimating, it will by definition take more time to do something. The more time you put into the estimate the more time will be added to completing the task. Here you need to find the balance between certainty and effectiveness. How much is the estimate worth? And we all know that estimates, however much time was spent making them, are estimates and not promises.

Testing too much

Uncle Bob says he demands 100 % code coverage. What a terrible waste of time for most projects! If you do pacemakers or deal with nuclear devices, fine, but if you do a corporate web site? Hell no! Test logic, test states, but don't test that the text you enter in the CMS show up on the web page - you'd notice if it didn't - and no one would be harmed if it failed to!

Unit tests are great in so many ways, but you need to keep a good balance between safety and productivity. I'm a big fan of Unit Testing, don't get me wrong, it's just that I don't think you need to test everything. For example obvious things. Or tons of nuances. Find all paths through the code and make sure a test case cover them. Would ever a bug occur, add a test for it and learn from it what kind of test you were missing. Certainly a useful one.

Kent's tweet

Nor do I think you need to inject every single dependency. If you're in the top layer of the code, which will only be used in this specific project, it can be OK to call that static method in the core formatting library without making a wrapper that implements an interface that you can inject. And you do not need to inject a wrapper of the CultureInfo class to make sure that your test will still work if you'd suddenly would change culture (unless you have a geographically spread project of course).

Kent Beck sent out an excellent tweet a while ago; "First you learn the value of abstraction, then you learn the cost of abstraction, then you're ready to engineer". In the first 10 minutes it was retweeted by 91 and favorite by 24!

Conclusion

Agile is good in many ways, but agile doesn’t necessarily make you go faster. As a matter of fact it may very well slow you down if you’re not careful! By paying close attention to what you do and what value you get from it you will be able to be more efficient by being less agile. Or rather, being the right kind of agile. The right level is no fixed though, so I can't tell you that. I just urge you to be observant and find the right level for you.

The Best Agile Methodology

I've been developing software for 18 years now, and I've been doing it in different agile ways for the last six years. Starting off with just having a daily stand up and soon after adding more of Scrum until we did the whole package. Later I have tried both XP and Kanban too, but none of them appear to be the perfect fit for me in my daily work. A year ago I read Crystal Clear by Alistair Cockburn and thought that might be what I was looking for... it was just that I wasn't sure how to fit the roles with my organization, so I dropped Alistair an email to get some guidance.

He's response was right on for me. He basically said that Crystal Clear is dead and pointed me to an article of his called "The end of methodology". Instead of trying to buy in on a whole package you should have a toolbox of good practices that you use to build the best methodology for your team in your current project with your current stakeholders. There's no way any off the shelf product will be the best choice, and most agile methodologies aren't even comprehensive enough to be called a methodology - it's just a set of practices that you need to complement with other practices to run a project.

That being said there are a few practices I'll never leave out.

1. First is the story. Be it a user story or a light version or even a use case, but something that explains the functionality we desire in words that make it valuable for the stakeholders. And then make that story 100 % done before going to the next feature. This is where I've had most problems before going Agile. Sitting with a project that is almost done in every single end and then having months of work just tying it all up.
2. Second is priority. That's really the essence of agile: Do what's most important first. If you do it in sprints, if you have burndown charts, if you code test first, if you have continuous integration - its all "processes and tools" and something we should value less. For every new story you start, ask if that's the most important thing to do - if this would be the choice if only thing could be added to what we already have.
3. Third is the retrospective meeting. Without that we won't achieve the continuous improvement that makes the process ever better. This is where problems are brought to surface so we can do something about them and make our team gel. This is also, not to forget, where we can scratch each others backs about all the good things we do, making the team gel even more.
4. Fourth is the demo. This is the developers moment of pride, and the product owners opportunity to make sure the assignment was carried out properly. Two good things make a right! Even a small bug fix may be understood incorrectly by the developer and it's a lot better to find it now than after the release. It's also a thousand times more time efficient to show what's been done and get immediate feedback that can be addressed and discussed than to just send over the software and wait for an email with comments. And if that isn't enough you should also see it as an opportunity to read each others satisfaction level.
5. And finally the fifth, and that's the stand up meeting. If it is a low pace project it might be enough with a weekly stand up, but most projects benefit from a daily stand up. But remember to keep it brief! This is not the time or place to discuss designs or what you did last night. Also make sure everyone shows up, and that they show up on time. And remember, this meeting is for the production team to coordinate today's work and bring up immediate problems, not a status meeting to report progress to the project manager.

That will set you up with a good base that is useful in every project! Then you'll probably add a number of different other practicies, patterns and tools that you should have in your toolbox. You find them in XP, Scrum, Kanban, Crystal Clear and other methodologies and apply them as you see fit. You can also find treasures to add outside the methodologies, like the Pomodoro Technique for example. Then you'll have The Best Agile Methodology - tailor made for you and your context!

Best of luck, and I would love if you took some time to comment my post!

My Top 20 #FailedTechBands

holly woolard ‏@holly_woolard
A Flock of SQLs #FailedTechBands

Dan North ‏@tastapod
JSON Donovan #FailedTechBands

Obinna Egbule ‏@Oegbule
The Black IPs #FailedTechBands

herr beesch ‏@herrBeesch
Run DMZ #FailedTechBands

MrLarry ‏@MrLarry
Rick ASCII #FailedTechBands

Tania ‏@CongoKasongo
Bit.ly Spears "It's Bit.ly b****!" #FailedTechBands

Christine Erickson ‏@christerickson
Linkedin Park #FailedTechBands

Dan North ‏@tastapod
Emerson Lake and PalmPilot #failedtechbands

Vince Speelman ‏@VinSpee
A Dell #FailedTechBands

Sarah ‏@SarahFKessler
Johnny Cache #failedtechbands

Christine Erickson ‏@christerickson
The Google Dolls #FailedTechBands

Mikolas Hämäläinen ‏@mikolas
Red Hat Chili Peppers #FailedTechBands

MissGalore ‏@MissGalore
Samantha Firefox #FailedTechBands

eremy Rosenberg ‏@lexlimo
The Grateful Thread #FailedTechBands

Rich Oglesby ‏@Rich_Oglesby
U2ube #failedtechbands

duckysherwood ‏@duckysherwood
Tori Cmos #failedTechBands

1126 ‏@1126tw
@DerGuteMoritz What about the Dead Lock Chili Peppers?#FailedTechBands

Jørgen Vig Jensen ‏@Kronsj
Iggy Pop3 #FailedTechBands

Scott Kerr ‏@scott_kerr
Dropbox Murphys #FailedTechBands

Alf Jørgen Bråtane ‏@alfjorgen
Perl Jam #FailedTechBands

Jim Halfpenny ‏@jimhalfpenny
System Of A Downtime #FailedTechBands

bgstaal ‏@bgstaal
Nine inch thumbnails #FailedTechBands

Tommy Bryntse ‏@tommycode
System is shut Down #FailedTechBands

Marco Tabini ‏@mtabini
.mobi #FailedTechBands

Dan North ‏@tastapod
The Console Twins #FailedTechBands

Retrieve multiple YouTube videos by ID

It took me quite a while to figure out how to get a list of YouTube clips from their ID's using the C# API. You could get all sorts of feeds, and query in all thinkable ways... except the most obvious thing, the ID.

Finally I found this little link and it was all very easy! Using the Batch method stuck me long before I found this, but it was not very intuitive to use. So this example made all the difference!

http://gdata-sharp.sourcearchive.com/documentation/1.4.0.2/classGoogle_1_1YouTube_1_1YouTubeRequest_f90a3aeef96701490ea5447db0b1501f.html

VirtualPathUtility.GetDirectory doesn't like query parameters

One of my customers have had some trouble with CSS files being cached in the browser for too long after a new version is released, making the site look bad. To resolve that problem I added a query parameter with the current revision to the end of the url, i.e. mystyle.css?rev=123. It worked fine on my computer, but when I sent it to the staging server I got the error "HttpException (0x80004005): '/my/path/mystyle.css?rev=123' is not a valid virtual path.".

After some research it turns out that the home made CSS compressor HttpHandler I'm using calls VirtualPathUtility.GetDirectory to get the directory of the CSS file. On my machine there was a flag set in the registry that allows for the characters ':', '?', '*'. in VirtualPathUtility.GetDirectory(string), on the server it wasn't set.

The flag is called VerificationCompatibility and found in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ASP.NET. If set to 1 it allows all characters but '\0' in the url, if not it will also disallow the three characters mentioned above. Since it has something to do with security my solution wasn't to set the flag on the production machine, but rather to send AbsolutePath to the method rather than PathAndQuery that was used first.

Since it took me a while to figure this out I thought it would be nice of me to let you know, and hopefully save you some time.

OutputCache on dynamically loaded UserControls

Yesterday I ran into an old bad bug in ASP.NET. When I put two controls with OutputCache set on the same page they rendered exactly the same - even tho they were different. The caching simply took one of them and showed on both places.

First I thought that is was an expected behaviour and that i needed to add some argument to the OutputCache directive, like VaryByControl or VaryByCustom, but I never managed to get that to work... I asked Google for help but couldn't really find any help. I did how ever find that the documentation said that adding two controls to the same page should automatically make them cache separately if not "Shared" was set to true in OutputCache.

That piece of information made me dig in another hole... Maybe it had to do with the fact that these controls were added dynamically, using Page.LoadControl and then added to a placeholder with ControlCollection.Add. I created a very simple test where Default.aspx had a placeholder in the body, and in OnInit I added two controls that only rendered a Guid - and had OutputCache. Adding them one by one didn't cause the bug, but when I added them in a loop it did! Wow!

Now that I knew what was the real problem i managed to get better answers from Google!

The first trace is as old as 2003, including hopes for it to be fixed in .NET 1.1 and .NET 2.0. The workaround suggested back then was to create a wrapping control without OutputCache that was the one that was dynamically loaded, and that had a static reference to the control you want to cache.

I wasn't really happy with that solution though and looked further. I found someone on StackOverflow that had the same problem and a reply leading to another solution, where you make sure that the stack is different on all calls. This would work around the problem, because the core of the problem appears to be that the Control you create with LoadControl is named by some hashkey from the callstack. That explains why it worked when I added the control on two separate lines in my test project!

I made a little test with a switch statement where every case did a LoadControl... but on separate rows. And it worked! Showing it to my collegaue Martin we started thinking about other solutions. First we did a recursion version, where we saved current recursion in Page.Items and made another recursion for every subsequent call. That worked too. Then we came to think of anonymous methods, but that didn't do the trick. Maybe they're named in the same way?

Instead we looked at DynamicMethod, which is the way we ended up using. At first I thought it would be bad for performance, but after implemention I did some profiling - and it was only half a millisecond per call extra! Here's the code if you'd like to solve the problem yourself:

public static class CacheFixer {
    private delegate Control LoadControlDelegate(TemplateControl page, string virtualPath);
    private static readonly Dictionary DelegateStore = new Dictionary();

    public static Control LoadControlWithCachingAllowed(TemplateControl page, string virtualPath, string key) {
        if (!DelegateStore.ContainsKey(key)) {
            DelegateStore[key] = CreateLoadControlDelegate();
        }
        return DelegateStore[key](page, virtualPath);
    }

    private static LoadControlDelegate CreateLoadControlDelegate() {
        var dynamicMethod = CreateDynamicMethod();
        return (LoadControlDelegate) dynamicMethod.CreateDelegate(typeof (LoadControlDelegate));
    }

    private static DynamicMethod CreateDynamicMethod() {
        var dynamicMethod = new DynamicMethod("", typeof(Control), new[] { typeof(TemplateControl), typeof(string) });
        var loadControlMethod = typeof (CacheFixer).GetMethod("LoadControlX");
        var ilGenerator = dynamicMethod.GetILGenerator();
        ilGenerator.Emit(OpCodes.Ldarg_0);
        ilGenerator.Emit(OpCodes.Ldarg_1);
        ilGenerator.Emit(OpCodes.Call, loadControlMethod);
        ilGenerator.Emit(OpCodes.Ret);
        return dynamicMethod;
    }

    public static Control LoadControlX(TemplateControl page, string virtualPath) {
        return page.LoadControl(virtualPath);
    }
}

Story Points vs Hours

In the Agile methods many suggest using something called Story Points instead of time when estimating the time a task would require for implementation. Rather than saying that it will take two days they would say it takes, for example, four Story Points. They then track what is called Velocity to see how many Story Points the team can complete in a week and suggest that about that many Points, let's say 20, is what the team can implement in a week.

I find this very awkward. Is it just a technique to hide the hours? Because, after all, most of us are still payed by the hour, and we also want to have an idea about the dates we might expect things to be finished. Story Points take this away. What would you say to the carpenter who you pay by the hour if he said he will have your bathroom done in 80 Story Points? He expects to burn about 20 Story Points per week, he adds. I would think he was a freak!

The only reasonable reason I find to use Story Points instead of Hours (or Days) is the notion that different coders may need different amounts of time to solve a task. With Story Points two coders may estimate a task to, for instance, four, while using days one might need two days and the other three days. That would implicitly say that coders are linear to each other in skill difference, since for the equation to sum up you'd need them to agree that the next task, estimated to two Story Points, would take the former one day and the later one and a half. Something I strongly disbelieve.

The more unreasonable reason is that as estimates are hard to do, you put a metric on them that is hard to understand to get away with them easier. If you say that something will take three days it's easy to see if you were right, while estimating it will take three Story Points will keep you safe. Who are to say how long a Story Point is? Hardcore agilists laughs at such a silly question.

I guess I'm missing something, but I have a hard time seeing the Point of Story Points. Feel free to enlighten me using the comment field!

p.s. I've never tried Story Points, so I might be totally wrong. It may be Gods gift to software developers. d.s.

EPiServer 6.0 compare bug

I've spent most of today, and a couple of hours yesterday, to figure out a weird problem with the function that allow you to compare two versions of a page in the EPiServer 6.0 edit mode. As soon as I clicked the compare button I got the YSOD telling me that a NullReferenceException was thrown from the method ComputeVaryCacheKey deep down in the ASP.NET rendering.

I hooked up the debugger and found that everything seemed to be in order. Altho, we had a few customized renderings in the chain so I started by disabling them one by one to find the cause. Once they were all removed I still had the problem tho, so I started on another track. Next thing I did was to look at the output caching used. Starting with simplifying the GetVaryByCustomString method in Global.asax so it should be almost impossible to have a problem there didn't help. Next thing was removing the OutputCache directives on my controls one by one and in the end - bingo! When the last OutputCache directive was removed I got some other error instead from the compare button.

So it comes down to that adding an OutputCache directive with VaryByCustom would make that ComputeVaryCacheKey crash when trying to compare two pages... I really need my output caching, but for the sake of coming to the bottom of this I left them out looking at the new error message. Maybe it could be the root course and if that was solved I could add the caching again?

The new error said "']]>' is not allowed within '<!CDATA[' blocks" or something like that. Well, fair enough. I looked at the source of the page being compared and there were a couple of CDATA blocks in the scripts generated by ASP.NET but none of them seemed to be malformed. At this point I decided to fire up another EPiServer site and see if the CDATA blocks looked the same, and if they would like being compared on that site. The CDATA blocks looked the same and they did like being compared in that context... Odd. Here I decided to add the OutputCache directive to a control on this site and see if it would work here - And it DID! There was a difference tho - this site was build on EPiServer 6 R2.

At this point I contacted the EPiServer support that within an hour reported back to me that this was a known bug in EPiServer 6.0 that had been solved in R2. The general advise was to upgrade to R2, but in case I couldn't do that I also was send a custom "CompareProvider" written by the support guy that should work with 6.0. So I put that in the bin directory, configured the site as he instructed me for activating this new provider and gave it a go. I still got that "']]>' is not allowed within '<!CDATA[' blocks"... but the good part was that I could add my OutputCache directives again and still getting that error!

Well, now at least I had my own custom code running closer to the problem. I reflected the provider that the support guy sent me and it was only one class. I copied the code to a class in my own project allowing me to set a breakpoint in there. When I had the code it was also obvious that the CDATA issue was addressed in it, but apparently it didn't address the problem well enough. I started by just removing the CDATA start and end block from the input to the compare function and then it worked! Or well, kind of. It didn't seem to have support for css being added with @import so I got a comparison of the page without styling... still, it was the best yet!

Hacking away even more I hardcoded an addition of the primary css to the source, and now it looked good too! A bit too hacky tho, for my taste, it struck me that I could reflect the source from the EPiServer 6 R2 binaries to see how they solved it there. The first line in that version was the Obsolete attribute - use "HtmlDiffProvider" instead it said. Wonder if that works with 6.0 I pondered, and reflected that source and added it to my project next to the code from the support guy. Then executed... and boom! That first NullReferenceException from the caching again! Dang. But, with the code from these two providers I managed to create a one provider that worked fine with EPiServer 6.0!

During this entire episode I did of course do a lot of searching on the web for guidance... getting none! Hence I decided to write this post to help the next person that enters this moat. It's not your fault! It's a bug in the platform! Stop wasting your time looking for problems in your project, and instead add your own CompareProvider.

If you can't reproduce it, you can't fix it

Every now and then you will be assigned a bug that seems impossible. You try to reproduce it, but you fail. You look at the code, and what happened seems impossible. It may now be tempting to add some if-statement to just make sure the object isn't null (for example) and close the bug. Don't Do That!

Doing that will cause a lot more trouble down the road. The problem will most likely come up again in some other part of your code, and then you'll be on yet one unplanned rescue mission. Maybe you'll be tempted once again to add a quick-fix and before you know your code will be full of them - all added with your best intention.

Then come the day when you find the real bug! Hooray! Finally your days of applying that dirty fix on different spots around your projects is over. But will you go back and revert your qd:s? No. Either there is no time, or you don't want to touch what works, or you just leave them there to take care about if this would happen again - but if you are like most people you will surely leave them there to rot. Don't Do That!

Leaving them mean more code to maintain, more things that can go wrong and more useless processor cycles. If there is an "if" in your fix (and it usually is) there will also be more forks in the code, which is very error prone.

So what should you do? You should put all your effort into being able to reproduce the bug. If you still fail, ask someone to help you reproduce the bug. If it seems to be a timing issue, write something that pushes your code so much that you cause the error within reasonable time before you try to solve it. Once again, how can you be sure you've solved the problem if you can't test if the problem disappeared when you're done?

Toedipping in ASP.NET MVC

Today I've finally had time to have a closer look at ASP.NET MVC, getting my hands dirty and actually doing something with it. I have vacation and the kids are at day care - so this was my chance!

I started up downloading the Visual Web Developer Express 2010 and created an ASP.NET MVC 2 template project. I executed it on the development web server, saw it worked, changed a couple of texts and reloaded to see my changes were applied. They were!

Then I googled to see if my web host, Binero, had support for ASP.NET MVC at all. I found that they had a blog entry with an instructional movie showing exactly how to make it work, so it shouldn't be a problem I figured.

I followed the instructions and uploaded my modified template site. It turned out the template site is using .NET 4.0 though, which Binero doesn't support, so it didn't work out of the box. Changing target version for the project to .NET 3.5 should do the trick I figured, so I did that and tried to run the project - and got some strange error. The page said Error CS1525 about a line in the View where the code was something about "Html.ActionLink". Google didn't quite help me, but pretty soon I noticed the obvious problem. It was marked "<%:" which is new in ASP.NET 4. Changing all "<%:" in the views to "<%=" made it work with the .NET 3.5 framework and I was back in game!

Playing around adding my own models, views and controllers was all pretty straight forward. Nowhere near as much magic as I had expected. I got a hold of it pretty fast and only had one thing stopping me so much I had to go googling again. Adding " - MySite" after the ContentPlaceHolder inside the title-element was not doing the trick for some reason.

<title><asp:ContentPlaceHolder ID="TitleContent" runat="server" /> - MySite</title>

All I got in the title was the content from the placeholder... I first thought it was some update problem and started recompiling, reloading, refreshing, re-everything, but no luck. Finally I went to google who could help me instantly. It has to do with the head-tag having runat="server" set. Full description and solution can be found here: TipJar: Title Tags and Master Pages

A couple of hours resulted in a very simple web site, but never the less a ASP.NET MVC web site.

My first!

How to commit code

When you commit (check in) your code to the source control system there are some things you should do to ensure quality and trackability.

1) If your source control system supports change sets, that is commiting a set of files as a bundle, make sure you just include one issue in that bunch. You shouldn't fix a bunch of things in all ends of the project and then commit them all in the same change set. The change set may well span over multiple projects tho, because one change set should include all the changes done to resolve that particular issue. A change set should also be compilable upon commit and not rely on the next one to be able to work.

2) Before you commit your change set you should diff every file against the repository version and see that you only commit things that were intended to commit. It's pretty easy to commit code you commented out, temporary variable names or debugging code if you don't review your own commits. If your commit is in central parts of the application or very large it is good to have another team member sit next to you when reviewing the changes.

3) When you've limited your change set to include only one issue and reviewed all the changes done, you should write a short description of the content in your change set. This is written as the "commit comment" and will be visible when you look at the log for your repository. Since your change set only should deal with one issue it is easy to write a brief description of what you've done. It's also good to include an issue id if you have an issue tracking system.

I guess many readers might think that this takes a lot of time - but think of all the time you save due to the higher quality instead! I've done this with all commits for many years now and it is very uncommon that I add bad code to the repository. It's not at all uncommon that I notice bad code while doing my personal code review upon commit tho!

My biggest problem when converting to this more professional approach was to limit my changes to only one issue. Yet today there are times when I can't commit only one change as I've fixed two (or more) issues in parallell without commiting the first one - and when they touch the same file it isn't possible to have only one change in the change set. If one of the fixes are small I usually solve this by reverting the changes for that issue temporarily while commiting the first change and then redo the changes for the next commit - but once in a while I need to write a commit comment with the dreaded word "... and ...".

I love deleting code

Yesterday I read a tweet saying "the next best thing after writing code is deletig code". My response was fast, saying "personally i like deleting code more. ;)". Let me elaborate on that!

Deleting code means that you either:

• found a better way to do something
• found unneeded abstractions
  
• found unneeded functionality

So, deleting code (at least when it's done on purpose ;)) is always done because you don't need it. Taking away code that you don't need is great, because the less code you have, the less can go wrong, and after the code is removed there is less to test and less to maintain.

Therefore, always strive to have as little code as possible doing the job. As they say in Extreme Programming, "Pay as you go: Build just enough to meet today's requirements".

You could get sad when removing code because it means that you or someone else have done something that could be considered a waste. Well, sometimes it was a waste but it won't be less a waste because you keep it. Most times the code you're about to delete served a purpose though, leading you to find the better solution.

So summing it up; Don't be sad about deleting code, love it like I do!

Odd way to fill your mailbox

On a site of mine I get an email every time an invalid url is requested. The purpose of that is mainly to find broken links. Sometimes a misconfigured crawler may spam me with a hundred mails, but they're pretty easy to delete and it doesn't happen frequently.

Tonight however, from 21.08 to 21.53, I received more than 3.500 such, all from the same ip. It would probably have been more if i didn't block that ip at 21.53, as I luckily was at the computer. The requested url:s were all directories and pages that exist on the site, but combined in odd ways, primarily stacking directories in long chains that doesn't exist.

As I don't have access to the firewalls of my hosting company I had to figure out a way to block the weirdo myself. My solution was to just terminate requests from that host in my asp.net page like this (but the real ip instead of the x:es):

  protected void Page_Load(object sender, EventArgs e)
  {
      if (Request.UserHostAddress == "x.x.x.x") {
          Response.End();
          return;
      }
      ...
  }