Usability Status Update - now with images and video


#1

Hello again everyone! Sorry I’ve been quiet the past few weeks–been busy with other things. I just wanted to give a little update/recap of my progress with the PocketCHIP’s usability. This is kind of a lengthy post, but hopefully it’s worth the read.

To start (for those who haven’t seen the other threads), I have a few issues with the PocketCHIP, and I know I’m not alone in the frustration of finding it less usable than expected. While there are a few others, the four primary problems for me are the screen, the keyboard, the interface and the lack of a mouse. Below, I describe the issues and how far I’ve managed to get in addressing them. Hopefully some of these things will be helpful to others, and I welcome any additional pointers you have for me as well! At the end I’ll have a couple of useful tricks too, so do scroll down please.

The Screen

The essential problem of the screen is that it is a lower resolution than pretty much any graphical program one would want to run is designed for. The result is that windows are either larger than the physical screen, or compacted in such a way as to be unusable. For some windows, it’s possible to hold down a key and drag the window around, but it’s awkward and doesn’t work on everything.

What I’d like is to be able to do either of two things: shrink everything (zoom out), and then zoom in on specific areas–something I can do to control my desktop from my phone–or allow the desktop area to be larger than the physical screen and let me pan around it freely.

This sort of thing is supposedly doable in linux using xrandr to create a virtual screen that you can either scale or pan around, but I haven’t been successful in getting it to work. It seems to come down to two things: not having a config file, and not knowing the hardware/driver information. I’ve gotten as far as being able to create another mode, but I can’t switch to it.

I know the CHIP can handle larger resolutions, as logging into it with a VNC client gives a much more manageable screen size. The closest I’ve gotten to reaching the level I’d like is to start the VNC server, then use a client on the PocketCHIP to log into itself. It’s a workaround, but it does work. See below under ‘interface’ for my current setup.

My thread for this topic is here: Screen size thoughts and questions

The Keyboard

I know that the developers went through a few iterations before settling on the keyboard design, but I think it could have stood a little more thought. For one thing, as a number of people have pointed out on the forums, the keys are hard to press for continuous use, and they seem to be unreliable. As far as comfort goes, I and few others have been working on overlays, or button covers that will make the keyboard easier to work with as well as provide for custom layouts such as game controls. Check out this thread for more on these projects. My own work on this has gone pretty well, but it’s delayed because I want to add additional buttons, as you’ll see in the next section.

Aside from comfort, the other problem with the keyboard is the way certain keys are accessed. For instance, to switch terminals in linux the common thing to do is crtl+alt+f1. However, since the PocketCHIP’s F keys are secondary to the number keys, it would have to be ctrl+alt+fn+1, which is both difficult to do (holding down four of these bubble-keys) and doesn’t work anyway since it’s the wrong keystroke. I’ve tried using xdotool to circumvent this but it doesn’t work. You can do it by plugging in an external keyboard, or by installing a program that works from the command line, which I’ll describe in the “tips” section.

The Mouse

The touchsreen works nicely, but it’s insufficient for use with a general linux gui environment. Every tap is the equivalent of holding down the mouse button, which means you can’t hover over menus or areas without dragging things around. Also, many programs and window managers are designed for mouse use, meaning they expect the user to be able to right-click or middle-click to be able to interact with them properly.

I have been able to create a sort of virtual mouse by using xdotool and xbindkeys, as described in this thread. By holding down the crtl key, I can move the mouse cursor around with the arrows, and I’ve set shift+backspace to trigger a right-click. It helps, but it’s still limited. I can’t do right-click-drag for instance. It would be nice if I could have it so tapping the screen is interpreted as holding down left, middle or right buttons depending on whether some other button is being held.

I’m working on adding physical buttons for mouse use which I’ll connect to the GPIO pins to use. I got a five-way surface mount switch (like a tiny joystick) to use for the mouse movement, and a few single buttons for pressing the keys. I’m still waiting for a breakout board and flux before I solder it up, and I expect that there are better ways to interact than the program I have in mind, but I haven’t been able to find much in the way of getting a key to be read as a direct keypress short of editing the kernel and I don’t know quite enough to do that yet.

The Interface

I see from the forum that there has been progress on customizing the home screen to add more pages of custom shortcuts, but it’s still a far cry from a typical desktop environment. As a few of us have found, however, it is possible to get to one of the built-in window managers (awesome) or install others. Once you launch one either by logging in as root or by doing sudo startx, the next time you boot up you’ll get the log-in window. This will only accept the root account for logging in, but you can select your environment on the top right corner. Enter the login information first, then select the environment, then hit login (if you select a different environment before you start typing it’ll revert back when you fill in the password). To “fix” this, and get back to the pocket-home world you need to switch to the terminal and enter the following line, as provided by this thread, then reboot.

chown chip:chip /home/chip/.Xauthority

HOWEVER, I discovered something interesting here while trying out different window managers. I switched to the terminal and fixed the permissions, then switched back to the other screen and logged in as chip! Now it logs me in straight to the window manager as the user every time–no more pocket-home. Interestingly, there’s a shortcut to the pocket-home program in the menu, so I can still pull it up if I want to adjust brightness, volume, etc., it’s just in a moveable, closable window now.

I looked around for what others do with small screens on linux, and found that the ones marketed for Raspberry Pi seemed to be using LXDE, so installed that. After altering a number of the settings for font size down to 6, it looks pretty good. Unfortunately there a couple snags–the shutdown and lock screens are still larger than the display, and programs still have the same problem described above, but it’s at least somewhat usable now. I haven’t found a way to unlock without rebooting when I’ve accidentally locked it though, so I try to avoid it. As a bonus, there’s a nice screensaver now!

Tips and Tricks

Here are few things I’ve picked up that I think will be helpful for others:

When you start a VNC server on the chip and log into it, you’ll find an empty screen and nothing to interact with. Go back to the terminal (either with SSH or on the PocketCHIP) and type export DISPLAY=:1, substituting whatever number your VNC server called the display for 1 (1 is the default, and it should tell you when it launches which one it is). Then, whatever you run after that (the file manager, for instance) will show up on the VNC screen. Similarly, if you’re connected remotely and want something to run on PocketCHIP’s screen, do export DISPLAY=:0 first.

It is possible to switch terminals from the command line without using keystrokes. The command is chvt 1, again substituting the 1 for whatever terminal you want. The GUI is on 6, by the way. You do have to install this first, though, but it’s easy to get. Just do sudo apt-get install kbd

It is also possible to perform a right-click by tapping and holding in place for a short while if you add the following lines to your 99-calibration.conf file. For me, the file is at /usr/share/X11/xorg.conf.d/99-calibration.conf and remember to use sudo to edit it!

Before the EndSection, add these three lines:
`Option "EmulateThirdButton" "1"
`Option "EmulateThirdButtonTimeout" "750"
`Option "EmulateThirdButtonThreshold" "30"

I found this trick here: https://learn.adafruit.com/adafruit-pitft-28-inch-resistive-touchscreen-display-raspberry-pi/extras

Running a different window manager means that the keyboard settings aren’t the same. I looked for a way to import the config file through the gui, but came up empty. You can, however, type this into the terminal to restore the PocketCHIP layout (and add it to the boot script to keep from having to do it each time). xmodmap /home/chip/.Xmodmap

For those new to linux, it’s useful to be able to find search for files using locate, but you need to establish the database it’ll look through. Once installed, use sudo updatedb to generate it.


That’s it for now, just wanted to say I didn’t disappear and report what I’ve learned. I’ll post pictures of the buttons once they’re up and running, and maybe some of my current desktop. Thanks everyone!


Adding buttons?
Adding buttons?
#2

I’ll post images (and video) later, but here’s a bit of mouse progress to report:

I attached a 5-way tactile switch to the bottom left of the PocketCHIP, and ran wires up to GPIO pins on top, then wrote a python “driver” that makes it operate like a mouse. It’s a little limited (can’t move diagonally, for instance), but it actually works pretty well, fits comfortably, and makes using programs much easier. The major downside is that it takes most of the GPIO pins to do it. I designed a second circuit that only needs three GPIO pins, but I’m trying to figure out the best way to place wires and a board or two inside the case. I’m thinking the best option would be to use some of those flat ribbon cables, but I’m having trouble identifying the right name for the cable and appropriate terminals. (there are a lot of varieties with different numbers of pins, sizes, etc, and its not clear what I need yet). If anyone can give me advice on this please let me know!

I did get a board that I think will work for connecting the top pins to one of those flat wires (once I get the appropriate header/terminal), but it’s on hold while I figure out exactly what connections to make.

Incidentally, I made a little “drop-in” board that slots into the bottom right hexagonal hole, with the plan of putting mouse buttons on it. It, too, is waiting for me to figure out the wiring in general, but it fits great.

I’m also planning on trying to use an i2c multiplexer chip so I can save using all the pins. I got a couple of the chips, but I don’t know how to work with them yet (got to read up on i2c and device trees).

Meanwhile, I’ve got a second method going as well. In order to facilitate more fluid control I got a PS2-style thumb stick. Last I looked at the documentation there isn’t a directly usable analog input to the CHIP yet, so I got a Trinket Pro. It sits inside the PocketCHIP’s case (attached by a 3M strip), and uses the USB interface. I’m planning on making a short USB cable to hook it up with less wiring, and maybe include a small hub so I don’t give up the USB port altogether.

I didn’t want to damage my PocketCHIP case, so I printed a new one with some extra holes in it to allow wires to pass in and out.

This leads me to a couple of open questions for everyone. How are you making connections to the PocketCHIP’s pins? Did you solder headers to the top row? Use clips? Does anyone know the names of the flat wires and their associated connectors to make wiring inside the case fit more neatly?

I’m toying with the idea of getting a custom PCB to handle the crossover between adding external buttons, etc, and the connections to the CHIP’s pins. Obviously, I’m not ready to do that just yet, but I’m wondering if anyone else would be interested. It’s cheaper to get multiples than single boards. Also, I’m new to designing PCBs (I’ve designed a few so far, but I’m no expert), so if anyone else has been planning to do something similar, please let me know!

That’s all for now, pictures to follow.


#3

As promised, here are some images of the Trinket/thumbstick mouse. It works, though it’s a bit too sensitive (easily dashing across the screen further than desired. I’m also not sure how comfortably it can fit alongside the PocketCHIP, even with a modified case to hold it in place.

Also, here’s a short video of the 5-way tactile switch in action. You can see that it fits pretty well on the bottom left of the PocketCHIP, and at the end I show how it’s wired up (this is obviously not the final version). As a bonus, you can see how my desktop looks with LXDE, and how the mouse works, including double-clicking to maximize windows. You can also see the pocket-home screen running as a windowed program, and how a couple of other programs look on the small screen.

PocketCHIP mouse in action!

I have managed to get I2C working, at least on 4.4, though it doesn’t seem like it can use interrupts like I’m doing for the mouse in the video. I am leaning towards the idea of making a custom pcb for the 5-way switch that will fit around the same place as shown in the video, with left, middle, and right buttons over on the right side. They’d connect to an MCP23017 on the back, inside the case, and on to the appropriate CHIP pins. I’m actually most of the way there. I’ve already designed the boards, but am trying to figure out which connectors to use to link them together (ideally something unobtrusive, like passing the wires through the pen/pencil holes. Nothing’s finalized yet, obviously. If someone posts how to do analog input I might try swapping out the 5-way switch for something like the flat PSP-style thumbstick.

I also still plan on finishing the keyboard overlay, but it’s on hold until I get the mouse part settled. Meanwhile it looks like a few others have had success with theirs!


#4

I did get windows to fit the screen size better with Matchbox window manager on PocketCHIP, and haven’t noticed keyboard issues yet.