In order to turn the ambition ideas of the Bon-App concept into a working application, two main technical challenges had to be tackled: Voice recognition and playback of video with a huge degree of control.
The solution for voice recognition had to offer reliable recognition of a small set of phrases (“Weiter”, ”Nochmal”, ”Pause”, ”Mehr”). As for video playback, special requirements came from the detail feature of Bon-App, i.e. the option to switch to a close up camera angle with the audio continuing seamlessly in the background. This made it necessary to find a video playback solution that offers frame-accurate seeking with very low latency.
Bon-App solved these issues by leveraging the power of the iOS AV Foundation framework and further more made it possible with Bon-App Recipe Format to create a way to describe an interactive recipe. But the team not only created the recipe format and the playback engine but further more definied a production process, that makes it possible to create most of the recipe description right within the video editing tool (AVID).