Blaine Bell, Ph.D
Principal Software Developer
Blaine Bell
_________
I build software and tools that specialize in graphics and user-interfaces. I have 15+ years of experience in building commercial software, a Ph.D in Computer Graphics, and extensive understanding on how to bring new ideas and current research into stable software with significant (100,000+) user-bases. My work extends a wide range of development tools, platforms, and hardware, using many different types of head-worn displays. I also have experience in porting large native code bases to different plugin architectures, including Unity, Microsoft Office, Qt, and HTML5/JavaScript, and comprehensive proficiency in developing mobile applications on any platform.

Programming

ABOUT ME
_________

I have more than 15 years of experience in building commercial software and have significant research contributions including 4 patents. My Ph.D thesis in Computer Science at Columbia University focused on Computer Graphics and introduced a new concept called View Management that addressed difficult layout and interaction issues in visualization and new types of interfaces, such as Virtual and Augmented Reality. At Schrödinger, I built commercial drug discovery software and focused on implementing leading-edge research in commercial 3D software. I was also the technical lead on a product called PyMOL, which is open source and has a significant (100,000+) user base. I was responsible for integrating and supporting products across every platform, I have ported large native code bases to different plugin architectures, and have comprehensive expertise in developing mobile applications on multiple platforms. As co-founder at Ravn this past year, I developed a battlefield multi-user head-mounted display (HMD) system that consists of many devices and provides the warfighter a platform to view video feeds and control many devices on the battlefield, including gun-mounted cameras, drones, robots, and facial recognition systems.

CONTACT INFORMATION
_________

Blaine Bell
Email:blaineblainebell.org

Full CV

PUBLICATIONS
_________
Patents
Feb 2016
Computer program composed from tabular storage structures that includes data and code
Sep 2012
Thin-client and distributed development using data programming
Jan 2010
System and method for view management in three dimensional space
Jul 2008
System and method for dynamic space management of a display space

Academic Publications

2007 Book Chapter
Representing and Processing Screen Space in Augmented Reality
2005 Thesis
View Management for Distributed User Interfaces
2005 Book Chapter
Maintaining Visibility Constraints for View Management in 3D User Interfaces
2002 Journal Article
Information at a glance
2002 UIST
An Annotated Situation-Awareness Aid for Augmented Reality
2001 UIST Best Paper
View Management for Virtual and Augmented Reality
2000 UIST
Dynamic Space Management for User Interfaces

Demos/Workshops

2005 Workshop
View Management for Distributed Display Environments
2004 Workshop
Visual end user configuration of hybrid user interfaces
2003 Workshop
Augmented Reality for Collaborative Exploration of Unfamiliar Environments
2001 SIGGRAPH Demo
Mobile Augmented Reality Systems
Other
2005 ISMAR
Immersive mixed-reality configuration of hybrid user interfaces
2005 Medinfo Conference
An evaluation of automatically generated briefings of patient status
2004 Demo
Visual end user configuration of hybrid user interfaces
2001-2002 Demos
Mobile Augmented Reality Systems
2001 Article
User Interface Management Techniques for Collaborative Mobile Augmented Reality

View Management
Data Programming

DATA PROGRAMMING
_________
A new software platform Data Programming (DP) allows developers to build distributed applications that run across all platforms and work together interactively. Essentially, the idea is: a program is an in-memory database. This database has unique features (see Patents) that allow developers to embed an entire program's logic directly into the database. Here is a video that shows some examples of it:

DP is not just another new programming language. It is fundamentally a new way of writing software across all platforms. I have currently built DP engines on IOS, Android, JavaScript, and desktop JAVA and C++, and it can easily be ported to any platform or device. I have also built an IDE (Integrated Development Environment) that shows how DP can be used to build distributed applications across mobile devices on the fly. There are many applications of this technology, including Virtual/Augmented Reality (VR/AR), IoT (Internet of Things), web app/mobile application development, and gaming.

DP has some significant advantages compared to current programming techniques:

I have developed many applications using DP, including all of my thesis work, where I initially developed the idea and used it to build complex rule-based graphical systems. I am currently looking for applications that can take advantage of DP and its unique qualities. I welcome you to read more about it, and to contact me if you are interested in learning more.

DATA PROGRAMMING EXAMPLES
_________
I have implemented many examples using DP, including my thesis work on View Management. Here I describe some prototypes that show how easy it is to develop complex applications that can run on many devices and work together interactively.

Restaurants
_________

This example is a application that suggests the streamlining of restaurant processes with mobile devices. The intention here is to show how easy it is to build these types of applications with many mobile devices working together, where devices can be added and removed at runtime. This does not suggest that waiters and chefs in restaurants will carry around mobile devices, but that it would be easy to build a system that could use other types of hardware, such as RFIDs attached to plates or mounted touch screens at the staging areas.

Way Finding
_________

People often visit unfamiliar physical spaces in which they want to access a particular item or go to a specific place. Having many screens embedded within the environment could help assist people to find their way. Here we demonstrate simple examples that could be part of a larger system, such as an indoor navigation system. The key here is that DP makes it easy to build the distributed part of the system so that the developers can focus on how the application will assist users by experimenting and finding the best use-cases for particular tasks.

Maps
_________

A tile-based map program, similar to many web-based maps, is demonstrated with additional features to support multiple popups without overlapping. This is done by using my unique Space Management algorithms to find layout positions dynamically. Click on the image below to see the interactive example of a Yelp query for pizza restaurants in a neighborhood in Chicago. (this works on mobile too)

VIEW MANAGEMENT
_________
We presented a new approach to designing user interfaces based on View Management, which is still extremely relevent to many applications, such as Virtual and Augmented Reality (VR/AR). View management refers to layout decisions that determine spatial relationships between objects, taking into account visibility constraints that allow applications to manage what users see. Our techniques are used to satisfy these constraints by controlling what is seen in a wide range of user interfaces: from 2D desktops to 3D immersive environments, from view-only presentations to interactive techniques, and from single user to collaborative situations.

I have recently implemented a plugin that allows commercial games and applications to take advantage of these View Management techniques. There has been many enhancements to these techniques, including the minimization of CPU utilization, scalability to handle larger environments, stability in annotation placement, and flexibility to change and control layout strategies on the fly. Here is a video that demonstrates this plugin:

The original View Management work was part of my Ph.D work and is introduced in this video:

We have also applied our techniques to augmented reality (AR), where information is visualized directly within the context of the real world by overlaying graphics onto what is seen. We develop tools, including an annotated situation-awareness aid based on a world in miniature, that make it easy to visualize information about the environment, including parts of the environment that might not be completely visible from their current location. Users can interact with the real world, in this case using a wireless mouse, to visualize information about the physical objects. These visual annotations can move freely between the physical world and the virtual objects based on the users' head orientation. Check out this video:

If you want more information about View Management, you can look at my publications as well as my old web page.

APPLICATIONS
_________
The current state of DP and its development environment shows its usefulness and viability for developers (See Development Environment). However, I am interested in using DP for commercial use to bring value to programmers and users that need it the most. I am actively seeking the best application(s) and are considering different industries such as:

Virtual/Augmented Reality (VR/AR)
_________

This idea of DP was initially developed when I was building VR/AR applications for my thesis work. The need for a platform that someone could walk into the room at anytime and be instantly connected to all devices including everyone else in the room is instrumental to building practical applications for Augmented Reality. DP provides this platform for developers to start building these kinds of applications without dealing with complex network libraries, APIs, or complex software infrastructure. I am considering the integration of my development environment into some existing tools, such as Unity, to assist developers in building these kinds of applications.

Internet of Things (IoT)
_________

Most of the software solutions for IoT focus on cloud-based solutions for making it easier to accumulate and manage a large number of devices. However, none of these solutions help program devices themselves, and how devices should communicate with other devices (including mobile devices) on the network. DP is the framework for building distributed software across all devices, so that as hardware gets more flexible/programmable, these devices should automatically detect other devices and work without necessarily being controlled from the cloud. This in-memory programmable database could implement hardware drivers and could help vendors make decisions when building resources like the CPU, memory, and power consumption, based on the complexity of the device and typical usage. The most important issue for IoT is to find a really good use-case that can showcase DP and bring value to the market.

Gaming
_________

A significant benefit to DP is that it is useful for building multi-player local games without necessarily communicating with a cloud. I think the most interesting opportunity could be in peer-to-peer games, where multiple players interact with multiple devices, such as board games with separate devices for players and shared board(s). These kinds of games are currently scarce and are very difficult to develop and maintain. DP makes it easier to build these types of applications. I am currently evaluating whether it makes sense to integrate it into the game development workflow (e.g., I am considering the integration into Unity, but am open to other platforms).

Web App/Mobile Development
_________

Building Apps on many platforms is difficult, and many businesses have separate development teams for each platform. There are some tools out there, such as Xamarin and Appcelerator, that allow developers to use one language that can be compiled into different platforms. However, this still requires a separate code-base for the server, which is typically written by another development team. DP allows developers to build both the client and server at the same time using the same code base.

DP could also assist developers in building a WYSIWYG (What-you-see-is-what-you-get) development tool directly in the DP development environment. These tools are typically great for building simple GUI interfaces, but when it comes to connecting the data in these interfaces with the rest of the program (including network communications), the models in other development environments break down. DP provides an environment in which all data and code (i.e., both the GUI elements and related data) exists, so that developers could build fully functional GUI's interactively for remote devices.

Digital Signage/Public Displays
_________

Digital screens are becoming more common in public, at events, and in commercial spaces. These displays are typically used for displaying simple pre-fabricated presentations or providing simple touch interfaces, such as to order at a venue or an informational kiosk. But what if screens were connected to each other and to everyones' mobile devices? What if a mobile application could provide an interface that could also show information on public displays in the same room? DP provides a development platform that makes it easier to build these types of applications, where all of the displays could be working together to provide a seamless experience for users. Some potential applications include wayfinding, presentations, improving social experiences and interactions by sharing digital information, or just simply improving existing digital signage by making it more interactive with everyone in the room.

Other Applications
_________

DP is a general new way of programming and can be used to build any type of application. Complex standalone applications with large code bases are particularly good for DP, since the tabular structures provide an easier way of thinking about code and modularization. Distributed applications such as drones and autonomous vehicles could also benefit from DP: communication with other devices throughout the different tasks will be manditory for regulation and safety. For example, drones that want to deliver something into a private residence or neighborhood will probably want to notify the residence of its destination, location, and consignment. Regardless of whether this system is cloud-based or local, it is difficult to build and maintain a communication protocol using APIs. DP and its distributed tabular structures provide an easy way for many devices to communicate and stream important information while also implementing the programming logic required to accomplish the tasks at hand.

Examples

DATA PROGRAMMING: HOW IT WORKS
_________
DP'S DEVELOPMENT ENVIRONMENT
_________

I have implemented the first development environment that is used to create applications on multiple devices at runtime. No other IDE can do this. There are significant features to this IDE:

Programming multiple devices at the same time is a difficult task, but DP's tabular structures make it easier to think about the distributed logic between all of the devices. It also allows everything in the program to be dynamically changed. Here are some screenshots of the IDE:

Main screen of IDE. Two significant parts of this interface include a view of the tables (upper-left) and a list of all devices currently connected (upper-right).
Table generation : Distributed tables can be defined at runtime with distribution strategies and relationships. These tables are visualized to help the developers understand the distributed logic of their program.

I have a working development environment that allow developers to implement applications like some of the examples described on this webpage.

Using the IDE, I can quickly demonstrate how to build a working client-server or peer-to-peer application that shows graphical objects and allows control of them from any device. This development is done with:

I also have more sophisticated examples that can also be developed using this IDE. If you are interested, please contact me, thanks!
Applications/ Direction
Examples
FAQ

FREQUENTLY ASKED QUESTIONS (FAQ)
_________

What is the difference between DP and Hot code swapping?
_________

Hot code swapping is a method of interactive programming that allows developers to change existing code at runtime. This method provides a good way to change functions for debugging, but typically has limitations when developers make structural changes to a program. In DP, all changes are supported during runtime. Instead of changing compiled code, the tables and code are changed directly in memory. For example, adding a member variable to a class is a structural change, which would require this variable to be added to all instances of that class. This is typically difficult for just-in-time compilers because the memory needs to be re-allocated and copied at runtime. In DP, this is analogous to adding a column to a table, which can be implemented in many different ways based on the DP Engine and platform.

What is the difference between DP and Reflection?
_________

Reflection is a mechanism inside a programming environment that provides a way to examine and modify the program structure and behavior at runtime. Unlike object-oriented reflection, DP provides a built-in way of querying and modifying an entire program structure, including tables, columns, rows, and table relationships. These entities are analogous to classes in OOP, where reflection is provided using explicit functions, such as getClass() or getMethods(). These explicit methods are typically not used to develop applications at runtime, but could be used occasionally for unique circumstances (and could alternatively be coded differently). DP Engines provide built-in reflection using system tables and functions, thus, querying these reflection properties and relationships is just like querying and changing other user-defined tables. This also allows DP to be extremely flexible and can support any operation at runtime (unlike OOP reflection, which typically has structural limitations, such as adding or removing a member variable from a class when other instances are already instantiated).

VIEW MANAGEMENT PLUGIN
_________
We have developed a tool for labeling and interacting with 3D environments that makes it easier to build 3D applications, focusing on the integration within existing commercial web pages. This tool is based on my thesis work, and can be used in any 3D application that wants to use screen space visibility for showing information directly within the 3D environment. Here is a video that shows some examples of how this plugin can be used in different types of applications, including 3D shopping and real estate:

3D SHOPPING
_________
We have developed a tool for labeling and interacting that makes it easier for companies to build online 3D shopping experiences. We are interested in finding companies that are interested in integrating these interactive 3D experiences into their current websites. Here is a video that shows how it works:

APARTMENT 3D
_________
We have developed a tool for labeling and interacting that makes it easier to build online 3D experiences. We are interested in finding interest to integrate these interactive 3D experiences into applications and websites. Here is a video that shows how it works:

HMD System for Military Use
_________
We have developed a military Head-Mounted Display System (HMD) for use on the battlefield. The goal of the system is provide operators with information about the environment and surroundings to avoid strenuous and dangerous situations while accomplishing their tasks. It is used as a tool that allows warfighters to see the enemy as fast as possible. We demonstrate some of the uses in this video:

VIEW MANAGEMENT PLUGIN
_________
We have developed a tool for labeling and interacting that makes it easier to build 3D experiences. Here is a video that shows how it can be applied to interactive 3D applications:
We have recently enhanced many of these algorithms, and are currently re-applying these techniques to Virtual and Augmented Reality for specific use-cases. If you are interested, please contact me.

© 2018 by Blaine Bell