Testlive Guide: Bug Reporting Information

Introduction

Table of Contents

Getting Started on Testlive

Guidelines of Testlive

The Test Server Community Forum

The Tracking System

Bug Reporting Information
- Bug Troubleshooting Methodology
- Bug Reporting Template
- Bug Reporting Methods

The Boost System

The Reward System

Reward Listings


Bug Troubleshooting Methodology

This method is about troubleshooting and narrowing down bugs that you notice in the game. Essentially, troubleshooting on the test server consists of the following steps:

  1. Verify unintended behaviour.
  2. Find list of situations that reproduce the behaviour.
  3. Formulate tests to find single element causing behaviour.
  4. Carry out tests to reduce the possible causes as far as possible.
  5. Report unintended behaviour and suggested cause as bug.


1. Verify unintended behaviour.
Sometimes this first step can be the hardest to really pin down. Often changes are released to Test with only a brief description of the intended behaviour. Sometimes, it has just never been clear what behaviour is intended. In general, you'll begin by noticing something happen that you feel may be wrong. Keep an eye out for:

  • Error messages in chat
  • Unusual Mob behaviour
  • Visual things that look unreal or broken
  • Differences between item descriptions and their actual effects
  • Mobs using abilities or doing damage that seem too powerful for their level
  • Items disappearing
  • Etc.

Try to imagine how things would be if they were real - and then anything that would seem not to 'fit' that experience may also be suspect. It's also important to look through patch notes and focus your attention in particular on the things that have changed. Actively try out things that you know have changed while keeping a sharp eye on how it is actually working.

Another source of changes to actively check on in a patch is often available in the databases on Auno and Anarchy Mainframe. Once the data for a patch is uploaded to these database, you can search for things that have changed in the patch. By looking through the items that have changed and comparing the details of those items to the old versions (also possible in both dbases) you can focus your attention on some elements of a patch that you know have recently changed - and as a result are more likely to be suspect.

Sometimes the database can tell you all you need to know about a bug. A spelling mistake, an incorrect requirement, a description of an effect that doesn't match the actual effect. These sorts of bugs are generally called "Data Errors" and need little troubleshooting - though it's wise to verify the error in game before reporting it. Sometimes what is in the database can be a little confusing - so always try to verify the data bug in game as well as in the database.

When you've found a behaviour that you think may be unintended, the next job is to verify as far as possible that it is in fact unintended. For this you'll need to do some research on what is intended:

  • Check the patch notes for the current patch
  • Ask in the Social channel on test.
  • Ask a CoT or BugHunter (BH)
  • Do a search on the AO forums
  • Check in the Bug Report threads in the Test Community forums
  • It can be useful to check a database like Auno or Anarchy Mainframe (Jayde's) to see what the exact effects are in the database.
  • Etc.

If you're lucky, you'll pin it down to one intended behaviour. If you're not so lucky, then you may get two or three opinions on what is actually intended. That's OK though, because you can still continue with the troubleshooting.


2. Find a list of situations that reproduce the behaviour.
This step is about finding out how broad the problem is. Ask yourself the following sorts of questions:

  • Does the behaviour always happen?
  • Does the behaviour happen when somebody else of the same profession does it?
  • Does the behaviour happen when somebody else of a different profession does it?
  • Does the behaviour happen only indoors?
  • Does the behaviour happen only in this playfield?
  • Does the behaviour happen only with this target/mob/NPC?
  • Does the behaviour happen with other similar nanos/items of different QLs?

These will give you an idea of the scope of the issue. The answers you get from these sorts of questions will also help you a great deal with the next step.


3. Formulate tests to find a single element causing the behaviour.
As you work out the scope of the issue, you'll usually begin to get an idea of what the possible causes there might be for an issue. What you are trying to find is a single common denominator that applies to all of the situations where the issue is reproducable. What one thing was the same whenever you were able to reproduce the problem?

Look for the following sorts of things in common:

  • Was it always with the same mob?
  • Was the same buff running on the player when the issue shows up?
  • Were the players wearing similar armour or equipment?
  • Were the players teamed at the time?
  • Were the players all of similar level?
  • Do the players all use the same Graphics card or system environment?
  • Did the players all do the same series of steps in the same order?
  • etc


4. Carry out tests to reduce the possible causes as far as possible.
When you think you've found a common element then formulate a test in the following way:

  • Find somebody who can reproduce the issue reliably.
  • Remove the common element that you think you've found.
  • See if removing that element stops the issue occuring.
  • Find somebody who has not been experiencing the issue.
  • Include the common element that you think is a cause.
  • See if including the element makes the issue occur.

So if you think a particular QL of an item is causing the problem. Find somebody who has that QL of item and see whether they have the issue. Then find somebody who hasn't seen the issue and give them that exact QL of item to test with.

What you are looking for is a reliable cause. During your testing, if the issue fails to show up when the cause is present, then it's likely that you haven't actually pinned down the actual cause. There are two possibilities in this case:

1) Incorrect Simple Cause:
There's a single simple cause of the issue, but you haven't found it yet. In this case, you'll need to go back through your other tests and look for other common elements.

2) Complex Cause:
Sometimes, issues are caused by two or more factors happening at the same time. Both factors need to be present for the problem to appear--but only one of the factors isn't enough. In this case too, you'll need to look back through your tests for other common elements.


5. Report unintended behaviour and suggested cause as bug.
Please see the following sections on the proper format for bug reporting and where to send bug reports.


Bug Reporting Template

Please use the following format when reporting bugs.

CHARACTER NAME(S)
Include your character's name (and names of those in group, if applicable).


TYPE OF ISSUE
Designate from the following, the most appropriate category for the bug report:

  • Animation
  • Client
  • Combat
  • General
  • Graphics
  • Interface
  • Items
  • Missions/Quests
  • NPCs and Monsters
  • Nanos
  • New Content
  • Organizations
  • Pet
  • Sounds and Music
  • Spelling
  • Tradeskills
  • Vehicles
  • Worldbuilding
  • Other


REPRODUCIBLE ISSUE
Reproducibilty is the key to being able to quickly fix a bug.

  • Always
  • Sometimes
  • Random
  • Have Not Tried
  • Unable To Duplicate
  • N/A


PLATFORM INFORMATION
Include as applicable:

  • OS
  • Video Card
  • Video Drivers
  • DirectX version

If not applicable, use: N/A


PATCH VERSION
The patch version information can be obtained from the in-game shift-F9 information. Example: 15.4.0.1.


SUMMARY OF ISSUE
Brief, short sentence, giving an overview of the issue.


DESCRIPTION OF ISSUE
A detailed description of the problem. Describe what you were doing, what you expected to happen, what should have happened, the error, possible ideas and suspicions. Include shift-F9 information as applicable.

Include:

  • The values of any skill or stat you think might contribute to the issue, if applicable. Include both base skill, and modified skill (implants, nano's, items, etc)
  • For issues with items or pets entering the Over Equip Limits that shouldn't be, be sure to include any information about armor, nanos, or otherwise, that you were using at the time to increase the stat in question.
  • If related to a specific playfield or mission, include all information received from shift-F9, and if possible in a screenshot.
  • A screenshot if the bug can be seen visually.


STEPS TO REPRODUCE ISSUE
Describe very thoroughly the detailed steps for reproducing the issue.


ADDITIONAL INFORMATION AND SCREENSHOTS
Comments and explanations regarding attached screenshots, as applicable. Before taking a screenshot, be sure to capture shift-F9 information. When including screenshots with bug reports which are e-mailed, please keep the total file size under 200k. You may crop the screenshot to only the needed info and also convert the screenshot to JPG format. An alternate method would be to upload the screenshots to a website, and provide the URL within the bug report.


Do not be afraid of providing too much information!
We would rather receive too much, than not enough.


Bug Reporting Methods

Non-Exploit Bugs:
E-Mail: bugs@ark.anarchy-online.com

  • Use only this e-mail address for reporting bugs found on Testlive. Please note that this address replaces all previous e-mail addresses used for Testlive bug reporting.
  • When including screenshots with bug reports, keep the total file size under 200k.
  • If in doubt as to whether something is a bug or exploit, treat it as an exploit, and e-mail exploits@anarchy-online.com.
  • NOTE: Use of the in-game /bug command will not count toward bug points.
  • A  convenient and easy-to-use on-line Bug Reporting Tool is also available. This tool provides all the neccesary fields that can be filled in easily, and is sent directly to bugs@ark.anarchy-online.com.

For Exploits Only:
E-Mail: exploits@anarchy-online.com

  • Include in the subject line: Test-Live Exploit
  • Include your character name, and any information you can to help reproduce the bug.
  • DO NOT speak about your discovery with anyone (other players, ARK-Bughunters, CoT, Funcom Devs, GMs) besides those from the ARK Exploit Team, who will have that title in their name.
  • Once you have reported an exploit, do not try to investigate it any further, nor use it to your advantage.
  • DO NOT email exploit reports to ark-testserver or testserver email addresses. Exploit reports are to be sent to exploits@anarchy-online.com only. Continued emailing of exploit reports to the wrong address will result in verbal, and written warnings, loss of points, and possibly exemption from the reward program in the future.