© 2020 Stefan Nussbaumer.
Creative Commons Lizenzvertrag
VideOSC

VideOSC – the interface

The default Screen

VideOSC, default screen
Fig. 1: The default screen

Figure 1 shows the default screen when you open VideOSC. On the right side you can see the toolbar with the following elements (from top to bottom):

  • The “play” button: Starts or stops sending OSC messages
  • The torch button: Switches device’ torch light on and off (only available if the backside camera is used)
  • The RGB mode button: Five different modes can be selected. Find more information on these modes further down.
  • The (manual) interaction button: Edit single or several pixels manually
  • The camera switch button: Select the backside or frontside camera (default: backside)
  • The info button: Display the current frame rate and zoom factor
  • The settings button
  • The quit button

Dependent on your device’ screen size and resolution some of these buttons may be hidden. In that case the toolbar can be scrolled vertically.

Tapping somewhere on the screen (except the toolbar) will hide the toolbar. Note that in order to interact with pixels or other elements than the toolbar the toolbar must be hidden.

Fig. 2: VideOSC with hidden toolbar

When the toolbar is hidden it can easily be made visible again by tapping on the right edge of the screen, respectively the the three vertically ordered dots next to the right edge.

On the bottom to the right you see the indicator bar with the following icons (from left to right):

  • The “play” indicator indicates whether VideOSC is currently sending OSC or not (in the screenshot it’s paused)
  • The RGB mode, either RGB or inverted RGB (negative). The screenshot shows that VideOSC is currently set to normal RGB mode
  • The interaction indicator (“no interaction” indicated in the screenshot)
  • The camera indicator (showing the backside camera is currently used)
  • The torch indicator (the screenshot showing that the torch is currently “off”)

The Settings

Fig. 3: VideOSC settings

The settings are hidden under the symbol in the previously described toolbar on the right of the screen.

Network Settings

network settings
Fig. 4: VideOSC network settings

Before you start playing with VideOSC you will have to set the IP address and port of the device to which you want to send OSC messages. By default the IP address will be set to 192.168.1.1 (which is often a router in a local network) and the port is 57120. You must find out the IP address of the machine you want to send OSC messages to and replace the default address with that address. Consult the documentation of the receiving device or application to find out under which port it expects to receive OSC messages.

“Device IP address” is the IP address of the device on which you have installed VideOSC. Note that in order to send and receive OSC both, your device running VideOSC and the client to which you want to send OSC messages, must be in the same network!

“Device port” is the port under which VideOSC can receive OSC messages itself.

“Root command name” allows you to uniquely identify OSC messages coming from VideOSC in situations where more than one instance of VideOSC is involved. E.g. the color information of the red channel of first pixel will normally be sent under the command name

1
/vosc/red1

By changing the root command name to e.g. “stefan” the full command name will be changed to
1
/stefan/red1

Resolution Settings

Fig. 5: VideOSC resolution settings

In the first line, consisting of two text fields, you can see and set the horizontal and vertical resolution. Changes will be picked up instantly, so you can even change the values while you are already playing.

Within the frame rate drop-down you can select among different combinations of minimum/maximum frame rates (depending on your device. Some devices may only provide one option). As a rule of thumb lower frame rates will provide a better exposure, allowing you to play in darker environments. When choosing a combination with a low minimum and a high maximum the camera will adjust to the light. Usually you should rather choose a low frame rate. Normally 15 frames per second should be totally sufficient (the frame rates are provided by the Android system itself and are not been implemented by VideOSC itself).

“Fix exposure temporarily” allows you to fix the frame rate if you have chosen a frame rate setting with variable frame rates. You will be forwarded to the following screen where you can either fix the frame rate by tapping the green ok_button button or cancel the action by tapping the orange cancel_button button.

fix exposure
Fig. 6: Fixing exposure

“Normalize output” will normalize output to floating point values between 0.0 and 1.0. Normally unedited pixels will generate integer values between 0 and 255 (eight bit range).

“Remember activated/deactivated/fixed pixels on quit” has not yet been implemented.

Sensor Settings

Not yet implemented

Debug Settings (temporary)

debug settings
Fig. 7: VideOSC debug settings

Debug setting will only apply until you restart VideOSC.

“Hide pixel image overlay”: Hide the super-low pixel image whose resolution is defined in resolution settings and show the original camera input. Note that the camera input is captured at the lowest possible resolution, so it might appear a little pixelated. Also the image’s aspect may not match the your device’s screen dimensions. Hence the image may appear stretched or clinched.

“Report the number of OSC bundles…” lets you determine whether all OSC bundles (grouped OSC messages that get sent once every with frame) that have been generated have actually been sent. Bundles for the red, green and blue channel get added a special OSC message, containing a counter for generated bundles and sent messages. The command names for listening to these messages and the values are

1
['/num_red_bundles', <num generated bundles>, <num sent bundles>]

1
['/num_green_bundles', <num generated bundles>, <num sent bundles>]

1
['/num_blue_bundles', <num generated bundles>, <num sent bundles>]

About VideOSC

A little background information on libraries and licenses.

RGB modes: Working with color channels

Tapping the icon in your toolbar will open the following pop-up:

RGB palette
Fig. 8: VideOSC RGB palette

The first two icons in this palette let you switch between normal RGB and inverted RGB aka “negative mode”. These two modes also have an impact on the values being sent through OSC. E.g. if a pixel has a value of 175 in the green channel in normal RGB mode, its value in “negative mode” will be 255-175 (i.e. black will become white and white will be black).

Whether you are currently in normal RGB mode or “negative” RGB mode will be indicated in the indicator bar at the bottom of your device’ screen and in the palette itself:

inverted RGB pallet
Fig. 9: Inverted VideOSC RGB palette

Icons three to five (red, green and blue icons) let you look at a single color channel. These modes will not change the output over OSC, yet they are useful as they let you edit a single color channel in a pixel as I will explain at the end of the next section.

Manual Interaction

By default VideOSC will not allow manual interaction with the pixels displayed your screen except for zooming. You can zoom in and out with two fingers by either spreading or closing them. The current zoom factor will be displayed by tapping the icon your toolbar:

Fig. 10: frame rate and zoom factor info at the bottom left corner of the screen

Manual interaction, which allows you to edit every single pixel and their OSC output, can be activated by tapping the icon in the toolbar. As soon as you have activated manual interaction all pixels will be numbered and an additional palette will appear:

manual interaction pallet
Fig. 11: Manual selection palette

Above screenshot shows the pixel selection palette in its default state. The third icon from the left symbolizes normal selection, meaning you can select arbitrary pixels on the screen by swiping over them. By tapping the green button on the right after you have set your selection will open the multi slider view discussed further down on this page.

The second icon from the left stands for the quick selection mode, meaning you will directly be forwarded to the multi slider editor as soon as you lift your finger from the screen. Within this view you will of course only be able to select adjacent pixels.

The leftmost icon stands for the eraser mode which allows you to instantly remove all edits from any pixel. Just swipe over the the regarding pixels and they will be reset to their original (merely camera-controlled) state immediately.

Like all palettes in VideOSC the pixel selection palette can be dragged to any position on the screen by tapping and dragging the icon on the right of the green confirmation button (a little hard to identity in above screenshot).

selected pixels
Fig. 12: VideOSC with selected pixels
Fig. 13: Multi slider editor

In above screenshot you can see the pixels that have been selected in figure 11. Note the numbers at at the bottom of each slider. These correspond with the pixel numbers that can be seen in the bottom left corner of each pixel in figure 11.

You will probably notice that there are two multi slider sets for each color channel, dividing the screen in two halves. If you look closer you will also notice that both halves are labeled in the top left corner: “values” and “mix”. This labels refer two the following:

  • “values” stands for manually set colors of a certain pixel. When you tap the green confirmation button in the selection palette (fig. 10) the previously selected pixels will be “frozen” to their current values. What you see in the “values” multi slider is the representation of their values at that moment. You can now manually change the color (and values sent over OSC) of one or more pixels through the slider bars. When you return back to VideOSC’s default screen (by tapping the green confirmation button) you will notice that the previously selected pixels are now fixed in their color (if that’s not immediately obvious cover the camera with your hand).
  • The “mix” multi sliders on the other hand allow mixing the manually set values with values coming from the camera. Setting a bar in the multi slider to 100% (like in the above screenshot) means that the value of the regarding pixel is set 100% manually. Setting it to 0% results in the opposite – value is set 100% by the camera. Any position in-between defines a mix between the two options. The position of the bar defines by which degree the value is defined through manual interaction vs. the value coming from the camera.
Fig. 14: VideOSC interface with edited pixels

When you’ve edited some pixels they can be identified by the small triangle in their bottom right corner. They may appear in different colors. These colors mean the following:

All channels of the pixel have been manually edited.
The red channel of the pixel has been manually edited.
The green channel of the pixel has been manually edited.
The blue channel of pixel has been manually edited.
The red and the green channel have been manually edited.
The red and the blue channel have been manually edited.
The green and the blue channel have been manually edited.

To edit only one color channel one may use the color palette (fig. 8 and 9) and select a single color channel.

As already noted switching to single color mode will not change the OSC output (like switching between RGB and inverted RGB does). A single color mode only allows you to edit a single color channel. When e.g. editing the red channel you will get a multi slider view like the following (having edited a single color channel of course has an impact on the values sent over OSC):

multi slider red
Fig. 15: Multi slider for the red channel

Resulting from this edit only values in the red channel are edited. Analogously, using the erase tool in the selection palette will only erase the edited information in the red channel.

Main Palette

Fig. 16: VideOSC main palette

Though this palette is visible in non-interactive and interactive mode it is mainly useful when you already edited some pixels. It allows you to take snapshots by tapping the camera icon. You will be presented a dialog, asking you to give the snapshot a name (you may leave it at the default one which is composed of the current date and time. I’m aware that the dark text color for naming the screenshot is not ideal but haven’t found a way to make it white yet…).

Fig. 17: Save snapshot

By saving the snapshot it will be stored in the list of snapshots that’s accessible through the folder icon (the second from the left in figure 15). The number of currently saved snapshots is displayed in the red circle that overlays the folder icon.

Fig. 18: The list of saved snapshots, accessible through the folder icon in the main palette

In the above list of snapshots you can do two things:

  1. Select a stored snapshots by tapping one of the listed snapshots. Pixels will immediately be switched to their edited (or non-edited) state as stored in the snapshot.
  2. Rename or delete a snapshot by long-tapping a snapshot. A dialog like the one displayed in figure 18 will open.
Fig. 19: Dialog for renaming or deleting a snapshot

In the list of snapshots you can see the number of pixels for which the snapshot was created (the number of pixels in the resolution at the moment when the snapshot was created). You need not worry about the snapshot becoming unusable when changing the resolution. If the new resolution is higher than the old one it will simply be amended by a number of unedited pixels up to the current number of pixels in the new resolution. If the new resolution is lower than the old one exceeding pixels in the snapshot will simply be ignored.

The entries “export snapsots set…” and “import snapshots set…” are not yet implemented.

Feedback OSC

Last but not least there’s on more icon in the main palette that needs some explanation: The OSC feedback button feedback OSC. When you send OSC messages from VideOSC to some client software or device it will sometimes be hard to remember which pixel controls what in the client. Hence, this is an attempt to solve the problem by not only sending OSC messages but also implementing the reception of OSC messages (sent to the IP address of device where VideOSC is installed under the “Device port” – see figure 4). These messages must follow the following naming convention:


1
'/<root cmd name>/<color pixel>/name'

To make it clearer, here’s a concrete example for the third pixel, addressing the blue channel. Upon receiving an OSC message under the name scheme described above, the string contained in the second slot of the message will be displayed directly within the pixel on your screen:

1
['/vosc/blue3/name', 'atk[2]']

Though I think this would be an important feature it must at the moment be considered as experimental. Currently my own music software doesn’t support it properly, so, unfortunately, I can only provide a screenshot from an older version of VideOSC, demonstrating in about how it should look like.

blue_channel_with_feedback
Fig. 20: The blue channel, displaying feedback (screenshot taken from VideOSC 1.0)