The cold shower that is Kongregate’s rating system
* I’m not one that likes rants so I made sure to include some positive bits at the end.
Before going further I think I need to make sure we all get some things:
- Kongregate seems to be doing just fine as it is right now. They look like a successful business so they are surely not in need of me telling them what to do.
- I always knew Golemizer was a niche game that few people would like. That was kinda the point of the project you know. To do something different.
I’m not sure why but I never really paid attention to the rating system on Kongregate before I realized that Golemizer’s exposure was directly related to it. So how is this system working?
- Players must create an account on Kongregate. While they are asking for your email it doesn’t ask you to validate the account using a link sent by email.
- Find a game.
- Click the stars.
- Receive 1 point for rating the game.
- Games features on the front page or receiving badges are choose based on the rating they received.
Seems just fine doesn’t it? The community decides what’s good and what isn’t. The “good” games are picked by Kongregate for further exposure to drag even more people to them.
So as a portal Kongregate makes sure that the first games anyone see on their website are the games that most people will likely play. Makes sense and Kongregate is successful so that’s probably good for them.
Where does Golemizer fits in this? Remember, I said Golemizer was a niche game so it can’t win a popularity contest. Put EVE Online and World of Warcraft in a popularity contest against each other, which do you think will win? Doesn’t mean EVE isn’t a great game but it’s still the loser. A lot of people won’t like Golemizer and that’s fine. I’m not doing this to be liked by everyone, that’s silly. I’m doing it for those who like this kind of game (and I’m still convinced there’s potentially enough of this kind of players so that one day Golemizer could become a full-time job).
So let’s say that in a day of 30,000 visitors on Kongregate only 200 players (0.6%) might be interested in Golemizer, how will they know the game exists? That’s the catch, they probably won’t unless everyone else that rated the game before were also interested in Golemizer (very unlikely).
So those who are not interested in the game are contributing to make sure those that might be interested will never hear about it. Well isn’t always the case? No.
Let’s say I tell you that Zombieland is an overrated movie that don’t deserve all the attention it gets. Isn’t it the same thing as giving 1 star to Golemizer on Kongregate? No because now you know there’s a movie called Zombieland. From there you might decide that if I say so it’s true so you’ll decide to never watch the movie based on what I said. Or maybe you happen to like movies involving zombies so you’ll give it a shot anyway. Or maybe you’re just curious to find out if the movie indeed deserves the attention it gets even if I told you no. But now you are aware that a movie called Zombieland exists.
If you give 1 star to Golemizer on Kongregate, you’re burying it into thousands of games. It’s rating will drop so Kongregate won’t consider to feature it on the front badge or to create badges for it. So there’s very little chances that you’ll be aware the game actually exists. Maybe it’s still not your kind of game but the only thing we are sure of here is that you will never play it just because you don’t know about it.
There are other ways to search for Golemizer on Kongregate. For example the “new games” section. It took about 12 hours before the game was out of it.
If you search per type of games then it is still ordered based on the rating so Golemizer is probably lost in the middle or the end of the list.
There is also the search engine where you can search for “mmorpg”. However this search engine search the entire website, forums included so there’s little chance you’ll find Golemizer this way.
Another interesting bit is that you receive 1 point for rating a game. It’s ridiculous to expect that everyone rating games will actually check all of them for more than 30 seconds. Just because the feature is there (and is used to set a level to your Kongregate account) you know people will just go through dozens of games in a minute just to receive their points. While it would be nice to expect such people to give 5 stars to every games that way we must be honest here and face they it’s just more tempting to give it 1 star. You know about that incredible sense of power the web gives you?
As I’m writing this the game has already disappeared from the “easy to find” spot. That’s 4 days after release on Kongregate.
Am I disappointed? Yes. Am I pissed about everyone that spent 30 seconds in the game and gave it 1 star? No, I don’t expect the majority to like Golemizer. Am I frustrated that this system is actually preventing Golemizer from being discovered by potential new players? Eh, yeah but I can’t do anything about it so I can only swallow the pill and deal with it.
One thing bugging me is the “no need to validate your account to vote”. Do all games that are well rated have been rated legitimately? Is there some kind of system to cross-check IP address with ratings? That might sound a bit jerkish to ask that but since the rating system has such a high impact of games I think it might be just fair to do so.
Am I saying that Kongregate sucks? No, like I said they are running a business and they seem to do pretty well so who am I to judge. I’m the one that needs them, not the other way around (though they would make money through me since I’m using their microtransaction API …). I’m just saying that unfortunately this is probably the wrong place for Golemizer based on the popularity contest that is the focus of this website.
I am now currently running a test ads campaign on Kongregate to help make people aware that the game exists. This can be just as bad as it can be good. I might just draw more 1 stars which won’t help the exposure of the game once the campaign is over and unless I’m really lucky I won’t have the money to keep advertising there. So if I run out of money to advertise it will probably reach a dead end.
Working on Golemizer as “Joe the developer” has been a roller-coaster ride since the start. Sometimes you’re feeling like you’ll take off in the sky and then you’re getting back to the ground. A 2 years and a half roller-coaster can play tricks on your stomach so it comes to a point where you might want to get off before throwing up on people around you (sorry for the terrible picture here). It doesn’t mean to burn the whole thing to the ground as THERE ARE people enjoying the ride but it might help to take a look elsewhere and come back once in a while just to check things out.
It’s still not all that bad as I didn’t lose my shirt in the process. But I have yet to make any profit yet. Sure the game is currently making just enough money to pay for the servers but it didn’t for several months. Add to this the money I spent in ads and artists work and I’m still a long way before buying myself a beer and be able to say “Ah! Thanks Golemizer for this fresh cold beer!”.
Maybe the situation will be different in 4, 5 years or more. Or maybe that I’ll just get lucky and from now on only people enjoying the game will rate it and it will reach the magical bar of 4 stars. Who knows fore sure. But like I said, got to watch your stomach before you’re unable to get back in the ride for good.
But to end on a positive note what could Kongregate do to give Golemizer a small chance?
Well, I’m using their microtransaction API meaning they get a share of the money I make when players are buying stuff in Golemizer. Could they give Golemizer a small temporary front page slot? Why not? It has the potential of paying up for them after all. If the game remains unknown then the only thing we know is that they will not make any money out of Golemizer.
Maybe there could have a feature called “Your pick of the day”. Based on preferences set by the user, each day 2-3 games are presented as their personalized pick of the day. If someone is interested in MMOs then at least once in a while Golemizer would get a decent exposure. Like I said it’s fine that people don’t like Golemizer but maybe others would like to know about it more easily than digging in hundreds of games.
Just with that I think it would give a fair chance to Golemizer on Kongregate without perverting the “users pick the high ranked games” system.
Anyone from Kongregate reading this?

about 3 years ago
The getting points for voting thing is rough. As a game designer, you have to understand what you’re rewarding when you give a reward. Giving points for giving a review or assigning some stars only rewards people for doing the click, not for putting any thought behind it. So, yeah, most games are going to get off-the-cuff ratings so people can increase their scores. A game that doesn’t look interesting from the loading screen, or that doesn’t reach out and grab you in the first 10 seconds. Unfortunately, Golemizer is one of those games that takes a bit to get into, so it’s not going to benefit under this type of system.
Ah, well. A bit of exposure is better than none.
(Quote)
about 3 years ago
Is there a possibility that we could ever see golemizer on newgrounds.com? Not sure if it would get better ratings than on Kongregate but between the new players on both it might help a little.
(Quote)
about 3 years ago
There are two paths you can go down if you want to play the Flash portal game:
- Test your games ahead of time and make sure they score at least a 4.0 out of 5. You can do this on New Grounds and you can do it by buying First Impression on Flash Game License. Or you can build a rating system directly into your game. Yes, most portal rating systems are horribly ineffective a discerning quality (see Newgrounds) but there is a bizarro popularity-based logic behind them once you get a hang of it. I find focusing a few weeks worth of effort on streamlining the first 30 seconds of gameplay helps immensely.
- Partner with a portal. Use their microtransaction system in a unique way, be a franchise they want to work with, etc. However, if you don’t have a game that wins the ratings game, your chances of partnering with a portal is extremely slim.
If your game can’t reach the front page via either of these techniques, you can
A) Update your game so that it does.
B) Make a different game
C) Not rely on portal distribution as a major part of your business model.
It’s a harsh world…<1% of games make it. The portal is selling the games on the front page…that is what they offer. They want to put the best and most broadly appealing games in those precious portal slots.
All the best!
Danc.
(Quote)
about 3 years ago
There’s one problem that makes it even more difficult for Golemizer to receive proper exposure. It’s not a Flash game the client is written in Javascript.
Kongregate is one of the few portals allowing non-Flash games to be integrated the same way Flash games are featured there. Newgrounds doesn’t allow that.
Like Brian said “a bit of exposure is better than none” and if I was able to drop Golemizer on a dozen of portals than the results might be pretty good actually even if the ratings wouldn’t be excellent.
I haven’t rely on portals so far and have been able to do quite some work to receive exposure even if things like the indie game press just won’t cover the game at all even though there was a time where 1 person building an MMO was something interesting (http://hanfordlemoore.com/v/how-one-man-made-an-mmo-an-interview-with-gene-endrody).
Eskil Steenberg is pulling it out with his project but I suspect he has a bit more history than me in the domain.
(Quote)
about 3 years ago
Oh and at one point I did have a partnership contract. The week before the game went live on the portal the company called me to tell me they were changing the focus of the portal.
Even though it wasn’t a huge portal (but comparing to the actual number of visitors the game receives it was huge) I would have been on the front page for quite some time. That would have been exposure I never received and I can only wonder what would have been the result.
I invested a lot of time and efforts for this deal to work and probably had to do so at the cost of improvements to the game or well, anything else.
So you must never say victory until you get the check in your pocket I guess!
(Quote)
about 3 years ago
I totally agree that portal rating systems screw over tons of great games. But they are a fact of life right now. They aren’t fair at all. Not at all. HOWEVER, they are predictable. Kongregate is a Flash game portal. Flash gamers are used to VERY intuitive gameplay, and they have short attention spans for learning games. Right now, successful Flash game developers spend as much time making their games intuitive and polished as they do on everything else about the game.
I tried Golemizer out very briefly and I would definitely not have put the game up on Kongregate in this state. Now I’m not saying I wouldn’t have LAUNCHED the game in this state (I certainly would have — these sorts of games should get launched ASAP so you can start gathering feedback). But since you only get one chance on Kongregate (and most other portals), I would hold off until the first five minutes of your gameplay are as good as they can get.
FlashGameLicense.com offers “First Impressions” which cost a dollar to get a random person to play your game for five minutes. They give you feedback, and you can instrument your game to collect raw data about how they played. There are probably technical limitations here that would make it unusable (it’s designed for pure Flash applications), but if possible I would definitely recommend it. (Full disclosure – I work for FlashGameLicense.com and actually implemented the First Impressions system! But it’s not a money-maker for FGL — the price barely covers our costs. We provide it as a service to help developers make better games.)
I have to reject your hypothesis that the main reason you were rated poorly is because this is a niche game that won’t appeal to lots of players. I think it’s much simpler than that. The kiss of death is that the game’s user interface is quite cumbersome and unintuitive to learn. Players need to have no trouble learning the game mechanics, and they need to have successfully done something cool within the first minute of gameplay. It took me way too long to figure out how to make a golem, and I gave up. (I forgot what the lady told me to do, and I couldn’t figure out the menus for training the golem skill, and I got bored and distracted.)
As a more concrete and simple example, when I logged in and clicked the pillar, nothing happened. My mouse cursor turned into a hand, the universal icon for “click to make it go”, and then nothing happened! Apparently I “selected” it, but there’s almost no feedback for that. I eventually discovered that I could right-click. So then I right-clicked Bob the Helper, and got THREE informational boxes instantly overlapping one another… the topmost one said “Not so fast!” as if I was being an idiot.
I realize that the way you implemented Golemizer limits your options and probably makes a lot of polish elements difficult or impossible. That’s very unfortunate because Kongregate users are used to playing Flash games with a lot of pizazz — and are used to having really intuitive user interfaces. On the other hand, I think that if you focus on getting players to understand the game ASAP, you will see big results, regardless of your tech limitations. (E.g. if you can’t make left-clicking work for some reason, at least there should be a guy on the first screen with a speech bubble that says “Right click things to interact with them!”)
I think the VAST majority of your visitors never find out what sort of game Golemizer is. The free game market is so competitive that if they can’t figure it out within a few minutes, tops, they will go find another game. I’ve talked with Greg from Kongregate, and he’s a great guy who definitely likes to help a niche game succeed. But he wouldn’t be able to help you with the game in this state… the interface is too much trouble to learn.
I want to end by saying that Golemizer looks pretty impressive, and from experience I can tell that thousands of hours have gone into this game. Since you’re currently struggling with player acquisition, though, I would take a step back and figure out what can be done to get players having fun as fast as humanly possible. Also, the Youtube video trick is great. But it should show me how to do cool things in Golemizer, or at the least, what is in store for me if I stick with it… it should definitely not be a slide show about the underlying tech!
I wish you luck! I think you’re right that in time it can have enough users to be a full-time job.
(Quote)
about 3 years ago
Thanks for the comment Eric. I always appreciate someone that takes the time to provide me that kind of feedback, may it be a player or someone with experience in this field.
I have to admit that part of my “frustration” is to expect maybe too much is so little time. What I mean is that Golemizer being an MMO I built on my free time squeezed between a day job, wife and just social life is a huge task (I don’t regret taking this path though). So the things you said are things I’m well aware of and a lot of it is related to me struggling to keep focus and motivation over a such long time span while keeping up with maintenance and everything that can cross the mind of an indie game developer.
You’re right to say that most visitors just don’t find out what kind of game Golemizer is and I can’t blame them. You pointed some things I could do to at least improve things a bit and surely that’s stuff I’ll be revisiting soon.
But still … (I know I’ll look stubborn here but that’s ok
that doesn’t mean I’m rejecting anything that’s been said here) my last stats are showing me an ARPU of $0.92 which really isn’t bad and the number of active players still have nearly doubled since the Kongregate integration. So while I know that by improving the game I would have a higher conversion rate it’s also telling me that it wouldn’t take that much of good exposure to make it big as hard as the game can be for visitors.
My conversion rate is terrible but the not so bad current numbers were still achieved with almost no exposure at all from any kind of significant source.
But what I clearly understand is that I’m in catch-22 here that only myself can fix. Game too rough = nobody interested to give it exposure. As much as I’d like to blame everyone about the situation I’m the only one that can provide a solution.
So basically I’m saying the same thing as you just with different colors maybe. By improving the game on the points you talked about I would first improve my conversion rate and then (hopefully) improve my chances of getting exposure. That might take some time though because of the reasons I’ve pointed before but it CAN be done.
Obviously I had to learn a lot for this project and I’m still learning a lot. That’s why I think this kind of posts are useful for me even though I might be making wrong assumptions because I’m lucky to receive comments from people with more experience than me and that’s a great way to learn.
Oh and about Kongregate I only have good things to say about the people there. I might not be having a good experience with the rating system but everyone I’ve talk to there has been amazingly helpful and friendly. They are really doing a great job.
(Quote)
about 2 years ago
Hi i’m zeme111 from the game and I’ve played about 3 months, I’ll tell you about what happened when I first started. I found your game on the MMOs section and at first thought it was a joke, I made my character look ridiculous on purpose and then later realized that it was permanent, I got in and was thinking in my head “I should just spam the chat for the lulz and go to another game” but I stuck with it and everything was good. My advice is COMPLETELY REDO THE TUTORIAL. Have the NPc’s talk in colored, bold, BIG text, keep the island small so the noobs can’t get lost, but have fighting right away and not walls of text! Make the sentences short and to the point, when they start they should have /attack and /follow already binded to their toolbar as default.
Perhaps, start in 1 zone *box pops up in top right corner* click on the ground to move! Right click Bob and hit talk!
Hi I’m bob! Golemizer is a game where you can do anything that your imagination can think of! Talk to my friend next to me to find out how to make a gnome!
Doc Toc is sitting there NOT HIDDEN IN A HOUSE *that’s an extremely common question on global is “where do i lrn teh gome skillz” and then have them make the gnome
Go north 1 zone, tell em how to whip out the gnome and then have multiple types of golems, not just zombies, maybe a blight horror, zombie, cow, rabbit… But make them all have 5hp and 1 power. And get them fighting from the first minute without these walls of text and cluttering arrows pointing to missing things and expecting you to find Doc Toc on your own. Make the buttons more sleek on the GUI. Instead of paying artists to make golems and furniture, make them get you a better GUI, you look at the GUI for 100% of the game, it’s very important to have it looking nice! And once they fight everything and get off tutorial island, tell them to spend their time points before they go anywhere else.
Short simple and informative tutorial is the key, with lots of flare.
~Zeme
(Quote)