I got and invitation to be a beta tester. How do should I do the testing?

After being accepted in the beta program, you need to register at least one device. Then you can install builds and provide feedback. These three tasks are described next.

Registering devices

ZotPad beta testing is managed with the TestFlight service. After receiving an invitation and accepting it, you need to register one or more devices with the service. Instructions for how to do this are provided by email after you have accepted the invitation. Registering a device is required for two reasons. First, all apps that are distributed outside the App Store must include a list of devices on which the app is allowed to run. If a device is is not listed in the app bundle, it will not install or run the app. Apple limits the number of new device registrations to 100 per year to enforce the App Store as the only distribution channel for apps. Second, TestFlight collects device identifiers, which are used to link crash and usage reports from the devices to particular testers.

You can register more than one device, but before doing this you should consider are you really going to use more than one device for testing. Because Apple limits the number of new device registrations to 100 per year, registering an additional device means that one less new tester can be accepted. So you should think carefully if you really need more than one test device.

Installing builds

A build is basically a snapshot of the current development version of ZotPad. Each build adds features and/or fixes problems from the earlier builds. During a beta test there are typically a few builds per week, but sometimes there can be breaks for even a few weeks. On the other hand, if a critical bug is discovered in a build, it is possible that several builds will be released on a single day. The devices that a build runs on is encoded in the build when the build is made. Because of this new testers cannot immediately access a build, but must be wait for the next new build.

Currently there are two different types of builds. Beta builds and release candicate builds. Beta builds have a distinct app icon, app identifier, and app name. These are illustrated in the table below. The main difference between the builds is that release candidate builds will install over the App Store version of ZotPad where as beta builds install on parallel as a separate app. Typically release candidate builds are used at the very late stages of beta testing to test the upgrading process or when testing a maintenance release.

Beta buildRelease candidate build

ZotPad beta
  • com.mronkko.zotpad-beta

ZotPad
  • com.mronkko.zotpad

Each build is identified by a build number (e.g. 1.2.1234). The first two numbers tell the version number  under which the version will be eventually released and the last number is just an incrementing counter without any particular meaning. Beta and release candidate builds have separate counters. 

Testers will be notified about new builds by email. Before testing it is good to check that you are running the latest build.

Providing feedback on the builds

Your role as a tester is to use the software as much a possible and provide feedback if something does not work or if you think that some feature should work differently. There are several ways to provide feedback:

  • If the application crashes, there is no need to provide any feedback at all. All crashes will be recorded by the TestFlight system and normally these crash reports contain sufficient information to identify the cause and to fix it. If more information is needed, you will be contacted by the developers.
  • If you want to propose a new feature or discuss an existing feature, you should do so through the UserVoice service.
  • All bugs are tracked at github. If you have an account for github, you can file reports directly.
  • If all this feels difficult, you can report problems by using the in-app contact support feature or simply just reply to the email that notifies about new builds
If the app crashes for you, this can mean one of the following two things: Your database might be out of date, or you have discovered a bug. Sometimes there may be changes in the database schema and the beta builds do not contain any code to upgrade between database schema versions. If this happens you can upgrade the database schema by simply using the "Reset data" switch in the ZotPad preferences. If the app crashes because of a bug, then there is no need for further testing of that build from you. (You may continue testing if you want.) The crash will be recorded by TestFlight automatically and the problem will most likely be fixed in the next build.

Obtaining a free license

Active beta testers are eligible for free ZotPad licenses. This is one way to thank the testers in helping improving the software. Unfortunately the TestFlight service does not provide any practical way of identifying active testers or distributing licenses. If you have been testing ZotPad and providing feedback for the software, you can get a free license by emailing zotpad@zotpad.com. If you think that you have been really active and helpful, and you can also request more licenses to share with your friends and colleagues.

The licenses are distributed as promo codes that can be redeemed in the App Store.

Another way to get a free copy of ZotPad is to just stay in the test team. All ZotPad versions that are available in the App Store will also be available through the TestFlight service.


Feedback and Knowledge Base