In-Depth
Silverlight, Windows Phone 7 and the Multi-Touch Thumb
For many Silverlight programmers, the most exciting news about Windows Phone 7 is its support for Silverlight as one of its two programming interfaces (the other is XNA). Not only can Silverlight programmers leverage their existing knowledge and skills in writing apps for the phone, but they should be able to build Silverlight programs for the Web and the phone that share code.
Of course, sharing code -- particularly UI code -- is rarely as easy as it first seems. The version of Silverlight used in the phone is called Silverlight for Windows Phone, and it's mostly a stripped-down implementation of Silverlight 3. When contemplating a shared-code app, you'll want to take a close look at the documentation: For each Silverlight class, the online documentation indicates which environments support that class. Within each class, lists of properties, methods and events use icons to indicate Windows Phone 7 support.
A Silverlight application for the Web gets user input through the keyboard, mouse and perhaps multi-touch. In a Windows Phone 7 program, multi-touch is the primary means of input. There's no mouse, and while there might be a hardware keyboard on the phone, Silverlight programs can rely only on the existence of a virtual keyboard -- the Software Input Panel, or SIP -- and only through the TextBox control.
If your existing Silverlight programs never directly obtain keyboard or mouse input and rely entirely on controls, you won't have to worry about the conversion to multi-touch. Also, if your programs contain their own mouse logic, you can actually retain that logic when porting the program to the phone.
On the phone, primary touch events are converted to mouse events, so your existing mouse logic should work fine. (A primary touch event is the entire activity of a finger that first touches the screen when no other fingers are in contact with the screen.)
Moving from the mouse to multi-touch will require some thought: Both Silverlight for the Web and Silverlight for Windows Phone support the static Touch.FrameReported event, but this event is a rather low-level interface to multi-touch. I focused on this event in my article "Finger Style: Exploring Multi-Touch Support in Silverlight" in the March 2010 issue of MSDN Magazine.
Silverlight for Windows Phone supports a subset of the Manipulation events that originated in the Surface SDK and have since become part of Windows Presentation Foundation (WPF). It's an example of how multi-touch is becoming more mainstream in steps. The phone supports only the translation and scaling functions, not rotation, and does not implement inertia, although sufficient information is available to implement inertia on your own. These Manipulation events are not yet supported in the Web version of Silverlight.
In summary, if you want to share code between Silverlight for the Web and Silverlight for Windows Phone, you'll be sticking either with mouse events or with Touch.FrameReported.
Consider the Thumb
However, there's another option: If you need only the translation support of the Manipulation events, and you don't want to worry about whether the input is coming from the mouse or touch, there is a control that provides this support in a very pure form. This control is the Thumb.
It's possible that you've never actually encountered the Thumb. The Thumb control is hidden away in the System.Windows.Controls.Primitives namespace and is primarily intended for ScrollBar and Slider templates. But you can also use it for other chores, and I've recently come to think of the Thumb as a high-level implementation of the translation feature of the Manipulation events.
Now, the Thumb isn't a truly "multi"-touch control -- it supports only one touch at a time. However, exploring the Thumb in some detail will give you an opportunity to experiment with supporting touch computing along with sharing code between a Silverlight application and a Windows Phone 7 application.
The Thumb defines three events:
- DragStarted is fired when the user first touches the control with a finger or mouse.
- DragDelta indicates movement of the mouse or finger relative to the screen.
- DragCompleted indicates the mouse or finger has lifted.
The DragDelta event is accompanied by event arguments with the properties HorizontalChange and VerticalChange that indicate the mouse or finger movement since the last event. You'll generally handle this event by adding the incremental changes to the X and Y properties of a Translate-Transform set to a RenderTransform property of some draggable element, or the Canvas.Left and Canvas.Top attached properties.
In its default state, the Thumb is rather plain. As with other controls, the HorizontalAlignment and VerticalAlignment properties are set to Stretch so the Thumb normally fills the area allowed for it. Otherwise, the Silverlight Thumb is just four pixels square. In Silverlight for Windows Phone, the Thumb is 48 pixels square, but visually it's really just 24 pixels square with a 12-pixel wide transparent border on all four sides.
At the very least, you'll probably want to set an explicit Height and Width on the Thumb. Figure 1 shows the Silverlight and Windows Phone versions side by side, with the default light-on-dark color theme of the phone. For both I've set the Height and Width to 72 and Background to Blue, which in the Silverlight version becomes a gradient that changes when the Thumb is pressed. Neither Thumb pays attention to the Foreground property.
[Click on image for larger view.] |
| Figure 1. The Silverlight and Windows Phone 7 thumb controls. |
Very often you'll want not only to resize the Thumb, but also to apply a ControlTemplate that redefines the control's visuals. This ControlTemplate can be extremely simple.
Sharing Controls
Suppose you want a simple control that lets the user drag bitmaps around the screen. A very easy approach is to put both an Image element and a Thumb in a single-cell Grid, with the Thumb the same size as the Image and overlaying it. If the ControlTemplate for the Thumb is just a transparent Rectangle, the Thumb is invisible but it still fires drag events.
Let's try to create such a control that's usable in both regular Silverlight and Windows Phone 7 projects. I'll assume you have the Windows Phone 7 Developer Tools installed. These tools allow you to create Windows Phone 7 projects from Visual Studio.
Begin by creating a regular Silverlight 4 project called Drag-Image. The resulting DragImage solution contains the customary DragImage project (which is the Silverlight program itself) and a DragImage.Web project (which hosts the Silverlight program in an HTML or ASP.NET page).
Next, add a new project of type Windows Phone Application to the solution. Call this project DragImage.Phone. (It's likely you won't want that name showing up in the program list of the phone or the phone emulator, so you can change the display name in the Title attribute of the App tag in the WMAppManifest.xml file.)
By right-clicking either the DragImage.Web project or the DragImage.Phone project, you'll get a context menu from which you can select Set as StartUp Project and run either the regular Silverlight program or the Windows Phone 7 program. A toolbar drop-down in Visual Studio lets you deploy the phone program to either an actual phone device or the phone emulator. (Visual Studio won't build the projects if this drop-down is set for Windows Phone 7 Device and no phone is attached.)
In the DragImage project (the regular Silverlight project), add a new item of type Silverlight User Control. Call it DraggableImage. As usual, Visual Studio creates DraggableImage.xaml and Draggable-Image.xaml.cs files for this control. Listing 1 shows DraggableImage.xaml with the visual tree of the control. The standard outer Grid named LayoutRoot will occupy the full dimensions of the control's container; the inner Grid is aligned at the upper-left corner, but there's a TranslateTransform assigned to its RenderTransform property to move it within the outer Grid. This inner Grid holds an Image element with a Thumb control on top with its Template property set to a visual tree containing only a transparent Rectangle.
Notice that the Source property of the Image element is bound to the Source property of the control itself. That property is defined in the DraggableImage.xaml.cs file shown in Listing 2. That file also processes the DragDelta event from the Thumb by changing the X and Y properties of the TranslateTransform.
To share that control with the Windows Phone 7 project, right-click the DragImage.Phone project and select Add | Existing Item to bring up the Add Existing Item dialog box. Navigate to the DragImage project directory. Select DraggableImage.xaml and DraggableImage.xaml.cs, but don't click the Add button. Instead, click the little arrow to the right of the Add button and select Add as Link. The files show up in the DragImage.Phone project with little arrows on the icons indicating that the files are shared between the two projects.
Now you can make changes to the DraggableImage files and both projects will use the revised versions.
To test it out, you'll need a bitmap. Store the bitmap in an Images directory within each of the projects. (You don't need to make copies of the bitmap; you can add the bitmap to the Images directory using a link.)
There should be two MainPage.xaml files floating around. One is from the regular Silverlight project and the other is from the Windows Phone 7 project. In MainPage.xaml for the Silverlight project, add an XML namespace binding called (traditionally) "local":
xmlns:local="clr-namespace:DragImage"
Now you can add DraggableImage to the page:
<Grid x:Name="LayoutRoot" Background="White">
<local:DraggableImage
Source="Images/BuzzAldrinOnTheMoon.png" />
</Grid>
The MainPage class for the Windows Phone 7 project is in a name-space called DragImage.Phone, but the shared DraggableImage class is in the namespace DragImage. You'll need an XML namespace binding for the DragImage namespace, which you can call "shared":
xmlns:shared="clr-namespace:DragImage"
Now you can add DraggableImage to the content area of the page:
<Grid x:Name="ContentPanel"
Grid.Row="1" Margin="12,0,12,0">
<shared:DraggableImage
Source="Images/BuzzAldrinOnTheMoon.png" />
</Grid>
That's probably the simplest way you can share a control between two Silverlight projects, one for the Web and one for Windows Phone 7. Because the control uses the Thumb, both programs work with the mouse or touch.
The downloadable code for the DragImage solution also includes a project named DragImage.Wpf, which is a WPF program that also uses this control. In the general case, however, sharing controls between Silverlight and WPF is harder than sharing controls between Silverlight and Windows Phone 7.
Color and Resolution
Aside from mouse and touch input, when attempting to share code between Silverlight and Windows Phone 7, you'll need to deal with two other issues: color and video resolution.
On the desktop, Silverlight displays black text on a white background. (However, a Silverlight program could use the SystemColors class in order to display the Windows colors selected by the user.) By default, Windows Phone 7 displays white text on a black background except if the user changes the color theme to display black on white. Windows Phone 7 provides handy, predefined resource keys, such as PhoneForegroundBrush and PhoneBackgroundBrush, to help a program use the selected color scheme.
Any code or markup shared between Silverlight and Windows Phone 7 that uses explicit colors will have to figure out some way to determine the platform on which it's running to get the correct colors.
The video resolution problem is a little trickier. All Silverlight coordinates are in units of pixels, and that rule applies to the phone as well. The average desktop video display probably has a resolution somewhere in the vicinity of 100 dots per inch (DPI). (For example, suppose a 21-inch video display handles 1600 x 1200 pixels, or 2000 pixels diagonally. That's a resolution of 105 DPI.) By default, Windows assumes that the display resolution is 96 DPI, although the user can change that to make the screen easier to read.
A Windows Phone 7 device has a screen that's 480 x 800 pixels with a diagonal of 933 pixels. Yet the screen measures only 3.5 inches diagonally, which means the resolution is about 264 DPI, some 2.75 times the resolution of the desktop display.
This means that shared elements of a particular size that look fine on the desktop are going to be too small on the phone. However, the viewing distance of the phone is usually shorter than for desktop displays, so the elements don't have to be increased by a full 2.75 times to be visible on the phone.
How big should the Thumb be for touch purposes? One criterion I've read indicates that touch targets should be 9 millimeters (or 0.25 inches) wide and high. On a desktop display with a resolution of 96 pixels to the inch, that's 34 pixels -- but on the phone it's 93 pixels.
On the other hand, the standard button on a Windows Phone 7 device is only 72 pixels tall, and that seems adequate. Perhaps the best approach is to experiment until you find something that's easy to use but isn't too clunky.
Making Adjustments
Traditionally, programs adjusted themselves for different platforms using preprocessor directives for conditional compilation. A Silverlight program defines the conditional compilation symbol SILVERLIGHT, and a Windows Phone 7 program defines both SILVERLIGHT and PHONE. (You can see these by selecting the Build tab on the project Properties page.) That means you can have code that looks something like this:
#if PHONE
// Code for Windows Phone 7
#else
// Code for regular Silverlight
#endif
Or, you can differentiate at run time by accessing the Environment.OSVersion object. If the Platform property is PlatformID.WinCE and the Version.Major property is 7 or greater, your code is running on a Windows Phone 7 device (or perhaps Windows Phone 8 or 9).
In theory, it's possible to define conditional sections of XAML files using the AlternateContent and Choice tags defined in the markup-compatibility (mc) namespace, but these tags don't seem to be supported in Silverlight.
But XAML can contain data bindings, and these bindings can reference different objects depending on the platform. XAML can also have StaticResource references that retrieve different objects for different platforms. It is this approach I used in the TextTransform program. I created the TextTransform solution the same way I created the DragImage solution. The solution has three projects: TextTransform (the Silverlight program), TextTransform.Web (the Web project to host the Silverlight program) and TextTransform.Phone (for Windows Phone 7).
In the Silverlight project, I then created a TextTransformer control that derives from UserControl. This control is shared between the Silverlight project and the Windows Phone 7 project. The TextTransformer control contains a hardcoded text string (the word "TEXT") surrounded by a Border with four Thumb controls at the corners. Moving a Thumb causes a non-affine transform to be applied to the Border and TextBlock. (It only works correctly if the quadrilateral formed by the Border has no concave corners.) The TextTransformer.xaml file doesn't create a new template for the Thumb, but it does define a Style as shown in Listing 3.
Notice the references to ThumbSize and HalfThumbOffset.
Although the TextBlock displaying the text gets the correct Foreground property through property inheritance, the Border must be explicitly colored with the same foreground color:
<Border Name="border"
BorderBrush="{StaticResource ForegroundBrush}"
BorderThickness="1">
Where are these resources defined? They're defined in App.xaml. The regular Silverlight project includes a Resources collection in its App.xaml file that contains the following:
<Application.Resources>
<SolidColorBrush x:Key="BackgroundBrush" Color="White" />
<SolidColorBrush x:Key="ForegroundBrush" Color="Black" />
<system:Double x:Key="ThumbSize">36</system:Double>
<system:Double x:Key="HalfThumbOffset">-18</system:Double>
</Application.Resources>
The App.xaml file for the Windows Phone 7 program references the predefined resources for the two brushes and defines larger ThumbSize and HalfThumbOffset values:
<Application.Resources>
<SolidColorBrush x:Key="BackgroundBrush"
Color="{StaticResource PhoneBackgroundColor}" />
<SolidColorBrush x:Key="ForegroundBrush"
Color="{StaticResource PhoneForegroundColor}" />
<system:Double x:Key="ThumbSize">96</system:Double>
<system:Double x:Key="HalfThumbOffset">-48</system:Double>
</Application.Resources>
Figure 5 shows the program running in the browser and Figure 6 shows the program running on the Windows Phone 7 emulator. The emulator is displayed at 50 percent of full size to compensate for the higher pixel density on the phone.
[Click on image for larger view.] |
| Figure 5. The TextTransform program in the browser. |
These techniques suggest that sharing code between the desktop and phone has become a reality.
[Click on image for larger view.] |
| Figure 6. The TextTransform program on the Windows Phone 7 emulator. |
If you want to delve a bit deeper into this subject, the Surface Toolkit for Windows Touch includes a SurfaceThumb control for WPF developers. This is just like the normal Thumb control, but it adds support for true multi-touch and events for when the thumb is flicked. For more information, see the Surface Toolkit for Windows Touch beta page.