PA Blog Update: Sprinting Along ; March 16th, 2013

Honestly not sure yet, but it's something I've been spending no small amount of time thinking about. Unfortunately, due to the amount of duplication that happens even when you have professionals entering bugs, we'll probably just use a bug forum, with some pretty strict rules on posting, so we can more easily priority filter the bugs that need the most attention and then delete threads as they get resolved. Not ideal, but with an alpha and beta that both are looking to have a huge user base, we'll need something flexible and easy enough to search through that nothing important gets missed. Forums are good for that, as they really lend themselves to the "squeaky wheel gets the grease" model. ;-)

http://www.mantisbt.org/demo/my_view_page.php

Have you considered using this? It would probably be much more efficient.

I've also used this tool in the past. Pretty intuitive and easy to use. Also its free, isn't it?

The thing I hate for that is it demands a separate login and I really don't want yet another one... =/

Unfortunately the current forum search engine isn't really up to that task. Even for casual searches I resort to Google.

Problems with the forum search engine:

1. Timeouts between unique searches

Sorry but you cannot use search at this time. Please try again in a few minutes.

2. Timeouts when browsing search result pages

Same as above, but more aggravating.

3. Not possible to search for exact phrases

Query: "Bacon planet" => search results includes posts that contain the words, but not necessarily the exact phrase, such as:

I vote for a bacon crusade! All us Baconeers shall rid the planet of all those who oppose our glorious substance.

4. Minimum and maximum character limit of search query

The following words in your search query were ignored because they are too common words: AI.
You must specify at least one word to search for. Each word must consist of at least 3 characters and must not contain more than 14 characters excluding wildcards.

---

Might as well replace the forum search engine with Google Site Search, but that lacks the following functionality:

1. Filter by author
2. Filter by subforum
3. Filter by exact date (although Google's time filter comes close)

Yikes, don't use a forum for anything except wordy discussions. There are forums which get used for bug tracking and mod release but it's really, really kludgey since they're not designed for it.

I agree. A forum is really not a very good solution to this.

One alternative I'm thinking about is actually a Google form. While not ideal, it has some advantages:
- Define a form that allows for any basic bug entry
- Make good use of pre-defined required fields so that it's easier to group bugs
- No login required - it's easy to make a form available to anyone.

Downsides are:
- It stores in a spreadsheet, which can become burdensome. But also is easy to clean out once old bugs are fixed.
- Subject to abuse. If someone started spamming garbage into it, could be a problem, but also fairly easy to ignore.

I've used Mantis, but I'm generally opposed to people having to sign up for even MORE accounts. The same reason I wouldn't want a JIRA extranet unless I could actually have it use a generalized login where everyone uses the same login. It's easy to restrict permissions on JIRA, but it doesn't handle huge numbers of people logging in on the same account all that well.

Keep freestyling though. I really like hearing all your thoughts on this. It's helping give me additional ideas and avenues to investigate.

I really don't care having another account..

Don't forget that the forum represents only a small percentage of those that backed PA. If the remaining backers don't have the time or don't care enough to post on the forums, would they be willing to dedicate themselves to a complex bug tracking system?

That said I definitely think that being able to identify those that send in specific bugs would be helpful especially since it would allow a the devs to gain information about the hardware/software profiles of the users. I'd also recommend integrating it into the game UI (might as well put that flexibility to use). One solution is to have the UI scripting take care of logging into the bug tracking system for the user. (Username would be the Ubernet username and password would be automatically generated but stored in plaintext so that the user can easily access it if they wish to log in outside of the game.)

The account issue could be solved with a single sign-on system. Or indeed not having unique user accounts at all if the bug tracker allows it - random/anonymous would be fine. The important thing is getting the bugs fixed, it's not a ranking ladder for bug hunters.

We already had some thoughts about the bug tracker in the backers forum:
viewtopic.php?p=679459#p679459

Might be better to continue the discussion about bug tracking in that topic.

Ding ding ding, I think we have a winner. A combination of a google forum integrated into our webkit to auto fill some needed information might be the best way to do this. I really like this idea though, and I have seen it done in a few games before, but with webkit, this could be both easy, low friction (no additional login needed AND already in game where the bug is), plus could even allow for easy attachments of logs if relevant to the bug.

I'd actually prefer this conversation stay in the public forum. I'd like as many points of view on this as possible, as it's an easy thing to do wrong, but harder to do right, and more people providing feedback is incredibly helpful.

If they don't want to dedicate a little to learn a simple bug tracking system, do you think that they will play the alpha to help finding bugs? I mean.. in that case they just want to see the game earlier.

As long as your bugtracking system aint bugged then ....

As someone with no knowledge of bug trackers and such, I would find it preferable to both be able to report a currently happening bug, point to a replay for a bug, and a catch all way to report bugs later (in case things crash a messy death).

I imagine you'd still have to use a browser to report crashes and webkit bugs...

Managing a horde of Alpha tester will be a pain, i already feel sad for UberTeam...

My 2 cents :
Force the use of a bug template.
(raw example : )
Title : WhatIsTheProblem
Detailed test steps
1/do this
2/do that
3/...
Expected final result
3/it does that !
Observed final result
3/it does this :-(

This contrain the testers to think about the issues they want to create, and spare devs a lot of pain to understand what the tester wanted to say and drasticly reduce reproduction time.

Ho, and allow to add a screen capture or small video. A picture is worth a thousand words.

Trust me, i'm a professionnal :ugeek:

(and i know, i know, templates are not valid for all cases.
But they will be OK for 90% of the bug encountered, so strongly worth the little sacrifice...)
(BTW, why am i posting this ? You are a compagny working with veteran of the industry, and i'm just me. Of course you already know that :-D )

Back in my misspent youth, I was a software test engineer. So while I want to leave some freedom for detailed bug reports, I want to automate mandate as much as possible. The forums will always be here for issues that need more detailed discussion, but the main goal of bug submissions is to get as much objective data as possible, with minimal creative writing required to get the point across.

I think integrating it into a button in the UI, either in game, pause the game, point to a place at the current time (or use the replay features to go back a bit?), and/or a button in the main menu/lobby that lets you point to a specific replay. Or just integrate the button into the replay GUI also...

Then a couple drop down menus, or text boxes, then give the user a number so they can refer to it if they need to go to the forums