Arduino + ESP8266 remote control LED using an Android App with Camera Live View

The Plan

In several of my previous post, I showed…

  1. …my setup to control LED’s using an Arduino + ESP8266. (LED’s Control Through ESP8266 + Arduino Web Page)
  2. …using the AppInventor environment to send a Web.PutText to send a message to the Arduino. (AppInventor2 Web Interface Trial)
  3. …embedding a live stream of a Foscam Wireless Camera into an AppInventor application. (Real time Wireless Camera view using AppInventor2)

My plan is to now mash up all three into one project.  I will make an app that will send Web.PutText to the Arduino through the ESP8266 to trigger LED light changes and have an embedded live Foscam Wireless window all on the screen on an Android App built in the AppInventor environment.

The Android App (built in AppInventor2)

The screen layout is made up of

  • 6 buttons to turn on/off 3 LED’s.  The buttons are just sending custom WebPutText keywords.
  • A WebViewer element which loads will load web page defined in text box element when the Go button is clicked.
  • Another text box to specify the web address of the Arduino + ESP8266 system.

Here is what is looks like….


The AppInventor code to point the Web component to the ESP8266/Arduino server will set the Web1 URL to point to the web address that is entered in the corresponding text box.  I was running out of screen space and didn’t want to have to add another button so I just have the code run after the address is typed into the box and something else on the screen is touched.  It looks like this…


The code to start up the live camera view is really just creating a Webviewer window which will display the web site specified in the corresponding text box whenever the Go button is clicked.  The rest is really up to the Foscam camera to serve up the live stream….


The LED controls are just sending some custom strings using the Web.PutText function.  I am using “PtcApp” as a keyword that is not likely to be seen by the server that I can key off of.  I follow up “PtcApp” keyword with a code for the {first letter of the LED color}:{1 for on / 0 for off}.  The magic will happen on the upcoming Arduino code.  Here is the AppInventor code….


The Arduino Code

I am only going to cover the additional Arduino code I add to implement the AppInventor functionality here.  The rest of the program has been detailed in many of my previous post.

The first step is really simple.  I just added another parser block to look for “PtcApp” in the incoming line from the ESP8266.  I left all the previous parsing blocks and function so that all the previous features will still continue working (ie: the web page interface is still fully functional).   Once the “PtcApp” is found, I call a function to parse out which color LED to control and if it is a request for On or OFF.  Here is the code snips….

 if (InLine.indexOf("PtcApp ") != -1) {
    CommandQue[QueIn++]=PUT_REQUEST;    //** See next section in post for what this is for **
//*** This parses out the LED control strings from custom Android App and sets the appropriate state vars *** 
void ParseCustomAppRequest(String InLine){
 if (InLine.indexOf("R:0")!=-1) RED_State=false;
 if (InLine.indexOf("G:0")!=-1) GREEN_State=false;
 if (InLine.indexOf("B:0")!=-1) BLUE_State=false;

 if (InLine.indexOf("R:1")!=-1) RED_State=true;
 if (InLine.indexOf("G:1")!=-1) GREEN_State=true;
 if (InLine.indexOf("B:1")!=-1) BLUE_State=true;

The First Test

This all actually started working very well initially.  Everything worked pretty much on the first attempt, but after 5 LED change requests from the Android App, the ESP8266+Arduino stopped seeing any more requests.

Debugging the logs, I found that each PUT request was holding an additional IPD channel open.  (In case you haven’t been following my previous posts, an IPD channel is the ESP8266 way of handling multiple TCP clients.  Each channel corresponds to a client so the server can send messages to a specific client).  The ESP8266 seems to have a limit to only be able to hold open 5 open IPD channels and then stop accepting any more connections.

I remembered another similar situation where the client wasn’t closing the connection.  It was because I wasn’t sending an HTTP header back for a GET request (see the “Extra Fix” section in my post “Arduino + ESP8266 Mini Server with Command Que“).  Hoping it was the same problem I made a very simple HTTP header with 0 length response and send it back each time I get a PUT request…

String PutResponse ="HTTP/1.0 200 OK \r\n"
 "Date: Fri, 31 Dec 1999 23:59:59 GMT\r\n"
 "Content-type: text/html\r\n"
 "Content-length: 0\r\n"
 "Connection: Closed\r\n"


    SendCIPChunk(PutResponse,CommandQueIPD_CH[QueOut]); // Send the CIPSEND command to respond to Put request 

Once I sent back the proper HTTP header, the client automatically closed the connection and all the requests from the AppInventor Web.PutText button clicks were coming in on IPD channel 0 consistently.  I was able to get the LED to change +100 times through a day of testing so I think it is pretty stable now.  🙂

BTW – I will include a link at the end of this post for the full Arduino code to get the complete picture.

The Demo

Here is a short video demoing the setup…

The Arduino Code

If you want the code…

AppInventor2 Web Interface Trial

The Plan

So far I have been able to demo using a simple TTY or a more complex HTML web page with an embedded form to control 3 LEDs connected to an Arduino through an ESP8266 wifi module.  I was looking for a way to control things directly from an Android App to make it look cooler and to allow for a more flexible messaging system.  In my last post, I also showed how I started messing around with MIT’s App Inventor 2 environment (Android Programming with AppInventor2).  I tried finding some sort of raw TCP IP connection or UDP packet sender in the App Inventor 2 tools, but couldn’t find anything.  The closest I could find was the Web.PutText and Web.PostText.  This blog post is to show the details of what I found trying to get familiar with this AppInventor feature.

The AppInventor Implementation

The AppInventor screen layout is very simple.  All I want to do at this point is to send either Web.PutText or Web.PostText so I just made 2 buttons on the screen that I can attach some code to.  I also added a Web component with the URL set to the IP address of my ESP8266 (see my previous post “Adding LCD to Web Page Controlled LED” for details on the ESP8266).  Here is a snapshot of the screen layout in AppInventor…


The “code” is also dead simple.

  • When Button1 is clicked, send a Web.PutText with a payload of “Hello put”
  • When Button2 is clicked, send a Web.PostText with a payload of “Hello post”

Here is what the “code” looks like…


The Results

I fired up my Arduino + ESP8266 system and launched my AppInventor application on my Android tablet.

When I click the button to test the Web.PutText feature, this is what the Arduino saw coming from the ESP8266…

  • +IPD,0,237:PUT / HTTP/1.1
    User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.2.1; M470BSA Build/JOP40D)
    Connection: Keep-Alive
    Accept-Encoding: gzip
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 9
    Hello put

When I clicked the button to test the Web.PostText feature, this is what the Arduino saw coming from the ESP8266…

  • +IPD,1,240:POST / HTTP/1.1
    User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.2.1; M470BSA Buil
    Connection: Keep-Alive
    Accept-Encoding: gzip
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 10
    Hello post

Both of these will send a full HTTP header before the text that I want to send.  So it isn’t exactly what I wanted like a TTY app which can send just the raw characters, but it still drastically cuts down on the data traffic vs. my previous implementation using a full HTML page with embedded form.  Using the HTML form required me to send a full HTML page just so a user could send back a POST command and then required me to send back another full HTTP header and HTML page.  Now I can build a fully customized Android app that can send a PUT or POST command without needing my server to send it something first.  Also don’t need to send back a full HTTP header and HTML to satisfy the browser.  Should make my system a little more robust when a user has the custom Android app.

The Next

Now that I have a better grasp of what raw data will be going through the ESP8266 when AppInventor calls the Web.PutText function, I will enhance my previous Arduino code to be able to control LED’s from messages coming from a custom Android app utilizing the Web.PutText component.

BTW – I contemplated if I should use the Web.PutText or Web.PostText component and decided to go with the Web.PutText for now.  The Web.PostText sends the string “POST / HTTP” which is identical to the trigger I use for controlling LEDs and sending back a full HTTP header and HTML page with the current LED states when using a HTML form.  The Web.PutText will send a string with “PUT / HTTP” which I am currently NOT looking for.  This will allow me to add to my current system instead of replacing my current system.

Hopefully my next post will be to demo a custom Android App controlling LED through the ESP8266 and Arduino from anywhere in the world where I can access the internet.

Adding LCD to Web Page Controlled LED (Arduino + ESP8266)

The Plan

In my previous post (LED’s Control Through ESP8266 + Arduino Web Page), I demonstrated how I used the ESP8266 and an Arduino Mega to create a web page that could control some LED lights from anyplace I could access the internet.  In my other previous post (Cheap $4.59 LCD with Arduino), I showed how I wired up a very cheap LCD to my Arduino and the demo software that showed how to draw various item and text on the screen.  I have not played around with the LCD a little and am now going to mash the 2 together and display a few key server statistics on the LCD screen.  Amazingly, it wasn’t as hard as I thought it would be.

In some of my previous posts, I already included details about how I have a counter for some key servers stats like number of web pages served, number or LED change request served, etc.  I am going to reuse those same counters and display to the screen.

The Arduino SW

First I draw the basic background of the screen by the following code.  I added comments to help make things easy to understand.

Note: I luckily guessed the LCD library was using the same color codes used for HTML pages.  I used this page to get the colors I wanted…

#define LCD_NETWORK_ROW 101
#define LCD_IPADDR_ROW 114
#define LCD_GRAY_ROW 101
#define LCD_SERVED_ROW 14
#define LCD_CHANGES_ROW 28
#define LCD_RESETS_ROW 42
void SetUpLCDBaseDisplay(){

 //*** Create blue title bar with name ***
 myGLCD.setColor(0, 0, 255); // Blue for interior of rectangle coming up
 myGLCD.fillRect(0, 0, 159, 13); // Draw a solid rectangle from upper left = (0,0) to bottom right (159,13) 
 myGLCD.setBackColor(0, 0, 255); // Blue background for upcoming text
 myGLCD.setColor(255, 255, 255); // White lettering for upcoming text
 myGLCD.print("Pete's ESP8266 Hack", RIGHT, 1); // Write the text right justified at row 1
 //*** Create gray bar at bottom with status ***
 myGLCD.setColor(64, 64, 64); // Grey 
 myGLCD.fillRect(0, LCD_GRAY_ROW, 159, 127); // Gray box at bottom of screen
 myGLCD.setBackColor(64, 64, 64); // Gray background for upcoming text
 myGLCD.setColor(255, 255, 255); // White lettering for upcoming text
 myGLCD.print("Network:", LEFT, LCD_NETWORK_ROW); 
 myGLCD.print("...init",RIGHT, LCD_NETWORK_ROW);
 myGLCD.print("IP Add:", LEFT, LCD_IPADDR_ROW);
 myGLCD.print("...init",RIGHT, LCD_IPADDR_ROW);

 //** Create labels for key server stats 
 myGLCD.setBackColor(0, 66, 0); // Darkish Green ... note: lettering is set to white from last above still
 myGLCD.print("LED Changes: " + String(int(NumberLEDRequest)), RIGHT, LCD_CHANGES_ROW);
 myGLCD.print("Web Served: " + String(int(NumberServed)), RIGHT, LCD_SERVED_ROW);
 myGLCD.print("ESP8266 RSTS: " + String(int(NumberOfResets)), RIGHT, LCD_RESETS_ROW); 

This is what the screen looks like ..


Whenever I increment any of my key server stats, I call this function…

void UpdateLCDStats(){
 myGLCD.setBackColor(0, 66, 0); // Darkish green background
 myGLCD.setColor(255, 255, 255); // White lettering
 myGLCD.print(String(int(NumberLEDRequest)), RIGHT, LCD_CHANGES_ROW);
 myGLCD.print(String(int(NumberServed)), RIGHT, LCD_SERVED_ROW);
 myGLCD.print(String(int(NumberOfResets)), RIGHT, LCD_RESETS_ROW);

This will update the LCD with the most recent counter values.  I was originally re-writing the entire line including the label, but found that I would sometime miss some of the incoming serial data.  The screen updating turns out to be pretty slow so writing all the characters must have been taking too long.  I minimized the amount of LCD update to just 3 digits to try to avoid this issue.

Here is the full Arduino sketch if you want it…

Save to your local drive and then you can open it in a text editor if you don’t have the Arduino IDE.

The Results

The testing went pretty well so here is a quick video of the system in action….

Adding LED States to Web Page (LED’s Control Through ESP8266 + Arduino Web Page)

The Plan

In my previous post demonstrated how I can remotely control some LEDs through a web page being served up by an Arduino hooked up to the ESP8266 (post here).  The web page was static and the check boxes always defaulted back to Off.  I am going to add code that will pre-populate the check box to show which state the LED’s are in when the web page is served up.  I also am going to add a new button that will request what is the latest state of the LED’s since multiple users could be controlling the LED’s.  This is what I want to see….HTMLFormSnap12_19

The HTML Implementation

To add 2 buttons that will return different strings to the server when clicked, I added this to my previous HTML…

   <input type="submit" name="LEDFormAction" value="Set LED States"> &nbsp &nbsp &nbsp 
   <input type="submit" name="LEDFormAction" value="Get LED States">"

You can find the rest of the HTML in my last post referenced above.

With this, the POST message sent to the server will also include:

  • LEDFormAction=Get+LED+States
  • or
  • LEDFormAction=Set+LED+States

Here is an example one for the POST messages sent when someone clicks the “Set LED States”…


Now all the web server code needs to do is look for a line with “LEDFormAction=Set+LED+State”  and then change the LED states based on the request found on the same line.  If it finds “LEDFormAction=Get+LED+State”, it will just formulate the HTML with latest LED states and send back without changing the LED states.  You can look at the code linked below to see the Arduino code implementation.

The Arduino Code Implementation

To pre-populate the buttons with the latest state, I just pre-formulate the HTML code for that section using an if statement like this…

 if (RED_State){
 RED_StateHTML = "<input type=\"radio\" name=\"RedLEDState\" value=\"RED_ON\" checked=\"checked\"> ON"
 "<input type=\"radio\" name=\"RedLEDState\" value=\"RED_OFF\"> OFF<br>";
 RED_StateHTML = "<input type=\"radio\" name=\"RedLEDState\" value=\"RED_ON\"> ON"
 "<input type=\"radio\" name=\"RedLEDState\" value=\"RED_OFF\" checked=\"checked\"> OFF<br>";
... repeat for each LED ...

Then I just concatenate the string into the full HTML code sent out and the requester will see the correct check box filled in.

To add the functions for the 2 buttons shown in the HTML above, I just look for the key words and do the appropriate calls as explained above.

 if (InLine.indexOf("LEDFormAction=Set+LED+States") != -1){
 if (InLine.indexOf("LEDFormAction=Get+LED+States") != -1){
    //Do nothing sinc the "POST / " command already is sending HTML page with latest LED states

The Arduino Code

If you want the full code…

Save to your local drive and then you can open it in a text editor if you don’t have the Arduino IDE.

BTW – I added a little extra code to actually try to wait for key responses during CIPSEND commands instead of a blind timeout in this version.  Not sure if it really helps as I still see the ESP8266 randomly just reset itself quite often but this seems cleaner way to do communications.

LED’s Control Through ESP8266 + Arduino Web Page

The Plan

In my previous post, “LED’s Control through TTY“, I showed how I can control some LED’s though a TTY client connected to my Arduino and ESP8266 ($4 Wifi module).  I also posted how I made a reasonably stable web server utilizing a command que to respond to html requests here “Mini Server with Command Queue Memory Issue“.  I am going to mash them together by adding a form in my HTML code that will send requests to the server to change the LED states.

The Web Page Implementation

I will use a HTML form to send a POST request back to the server.  If you are like me and not familiar with how this works, I will explain what I know and how I will try to get it to work.

First I found out how to create a form with radio buttons and a submit button.  Here is what the page looks like…


I found that helped me write the HTML code for the above like this…

<form action="" method="post">
 <legend>Red LED State</legend>
 <input type="radio" name="RedLEDState" value="RED_ON"> ON
 <input type="radio" name="RedLEDState" value="RED_OFF" checked="checked"> OFF<br>
 <legend>Green LED State</legend>
 <input type="radio" name="GreenLEDState" value="GREEN_ON"> ON
 <input type="radio" name="GreenLEDState" value="GREEN_OFF" checked="checked"> OFF<br>
 <legend>Blue LED State</legend>
 <input type="radio" name="BlueLEDState" value="BLUE_ON"> ON
 <input type="radio" name="BleuLEDState" value="BLUE_OFF" checked="checked"> OFF<br>
<input type="submit" value="Submit">

The Server Implementation

Disclaimer: I am no expert.  Just sharing what I found through googl’ing and trial and error.

When someone clicks the submit button in the form above, it will send a “POST” to the server.  This post will include the name / value pair as defined in the above.  Best to just show what my server sees when someone clicks the submit button above….

Connection: keep-alive
Content-Length: 64
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Linux; Android 4.1.2; SPH-L300 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.93 Mobile Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8


Note the last line highlighted in cyan.  In this example, the RED and GREEN LED ON button would have been checked and the BLUE LED OFF button was checked.  I plan on using this request line to control the LED states.  The one side effect is that the browser is now waiting for a HTML response.  My simple hack to address this is to just re-send the same web page again for now.  Eventually I will add code pre-populate the radio buttons with the current state of the LED’s instead of all OFF.

The Code

The really messy code is available here:

Save to your local drive and then you can open it in a text editor if you don’t have the Arduino ID.

The coding is REALLY sloppy for now.  Will go back to clean up once I get everything I want working.

The Hardware Setup

If you missed it in my earlier post…


The Demo

Video demo of the system in action…

Mini Server with Command Queue Memory Issue

The Problem

Previously in my post, “Arduino + ESP8266 Mini Server with Command Que“, I showed how I overcame the problem of missing incoming messages from the ESP8266 by implementing a command queue system.  As with most engineering issues I have worked on… once you solve one problem you make another.

The command queue system worked great until I started stuffing the queue entries with a lot of data. I wanted a much more elaborate HTML page which ended up being + 800 characters long.  That pretty much got me into a bunch of memory problems that took me a long time to debug.

The Behavior

Everything was working great….. for a while.  Then I added a form to my HTML code.  The first time it served up the HTML, everything was great.  The next time, nothing would be sent back out.  Every time my program tried to send out the long HTML string saved in the corresponding queue entry, there was nothing coming out.  I thought I had broken my code someplace in the loop and spent a long time trying to back track.  Eventually I found out that if I cut my HTML size down, things were ok again.  The only difference was the length of the string.  I am not 100% sure, but that sounds like a classic memory corruption issue and I know using a lot of String objects in Arduino code is risky on memory.

The Plan

I spent a lot of time trying to figure out how to change from using an array of Strings to and array of  char arrays.  Ended up being a dead end.  Instead I stepped back and thought about how to actually address the real problem… wasting memory.

Instead of storing the entire command string in the queue, I am only going to store the type of command that needs to be sent  and any critical info (ie: the +IPD channel from the ESP8266).  When the queue pointer gets to the command entry, I will then formulate the string and send to the Serial1 port.  This should dramatically reduce the amount of memory I am wasting.

The Implementation

It took a lot of debugging and testing and revising.  Ran into an issue with multi-part command, but here is the relevant part.  I first have some #defines for the different types of commands to make the switch statement easier….

#define HTML_REQUEST 1

In the module to enter commands into the queue…

 if (InLine.indexOf("GET / ") != -1) {
 if (InLine.indexOf("POST / ") != -1) {
 if (InLine.indexOf("favicon.ico") != -1) { 

Previously I will filling in an entire really long string when there was an HTML request that could be taking 100’s of bytes in memory multiplied by how many requests came in.  Now I am using only 1 byte in memory for each request.

Then in the module process the command…

 switch (CommandQue[QueOut]){
 float CIPSendTime;
 //*** Build the HTML code ***
 //Note: Need this first since HTTP Header needs length of content
 HTMLCode = "<HTML>"
 "<HEAD><TITLE>Pete's Mini8266 Server</TITLE>"
 "<BODY><H1>Welcome to Pete's ESP8266 \"hacking\" project</H1>"
 "<form action=\"\" method=\"post\">"
 "<legend>Red LED State</legend>"
 "<input type=\"radio\" name=\"RedLEDState\" value=\"RED_ON\"> ON"
 "<input type=\"radio\" name=\"RedLEDState\" value=\"RED_OFF\" checked=\"checked\"> OFF<br>"
 "<legend>Green LED State</legend>"
...  Finish off the rest of the HTML code ...
Serial1.println("AT+CIPSEND=" + String(CommandQueIPD_CH[QueOut]) + ","+ String(HTTPHeader.length()+2));

Now there is only one long string and is sent and memory freed right away.

There is a lot more hacking to be able to match up the IPD channel with the string and continue queuing up request between multi-step commands but too long and detailed to explain.  The really messy code is available here:

Save to your local drive and then you can open it in a text editor if you don’t have the Arduino ID.

The coding is REALLY sloppy for now.  Will go back to clean up once I get everything I want working.

The Testing

I have been hitting the mini-server from multiple clients with more than a 6 line web page and things seem to be working well.  I do get random ESP8266 resets but my recovery code has been able to re-init and get everything working again each time.

The Next Step

Going to move on and show how I am using an HTML form and the “post” mechanism to control LED’s on my board from a web page.

Arduino + ESP8266 Recovery Mechanisms

The Plan

I have spent a lot of time trying to make my Arduino + ESP8266 mini web server more stable.  I still run into some issues that I can’t avoid or recover from.  To address this, I am going to try to implement some auto-recovery mechanisms.

The Problems

  1. The ESP8266 seems to randomly just reset.  Could be power issues supplied from supplying the ESP8266 from the USB port or my bad web server code.  Not sure, but want to be able to recover from this situation.
  2. ESP8266 serial port gets stuck with “busy s…”.  I am not 100% sure, but it seems to consistently happen if I abort a HTML request from my browser (which closes the connection) right when my server code is trying to send the HTTP header and HTML code back.  After then I get the “busy s…” for anything I try to send to the ESP8266.

The Implementation

Random Reset: My serial port parser looks for the occurrence of “System Ready”.  This is sent by the ESP8266 any time it resets.  If the “System Ready” string is found, send out all the AT commands to start up the ESP8266 as a server.

if (InLine.indexOf("System Ready") != -1) {
 Serial.println("The ESP8266 Reset for some reason");

void InitWifiModule(){
 int CommandStep = 1;
 SendCommand("AT+RST", "Ready", true);

 SendCommand("AT+GMR", "OK", true);


 SendCommand("AT+CIFSR", "OK", true);




“busy s…” Lockup: Once I see this, I have never been able to recover the ESP8266.  Even sending the AT+RST command doesn’t help.  The only way I can find around this is to HW Reset the ESP8266.  Luckily there is a HW Reset pin on the ESP8266.  I wired digital IO 22 (randomly picked on my Arduino Mega board) to the ESP8266 through my NTE4050B “one way level shifter”.  Now when my parser sees “busy s…” it will toggle digital IO 22.  My random reset monitor (see above) will see the ESP8266 reset and will send the server init command.

if (InLine.indexOf("busy s...") != -1) {
 Serial.println("dead with busy s... HW Reset");
 QueOut=QueIn; //Clear out the command que
 // Note: Parser should see the reset and start the InitWifiModule routine

It has been a few posts since I posted a diagram.  Here is the latest with the Uno replaced by the Mega and the new ESP8266 reset signal in place….


The Testing

To simulate the random reset, I turned off/on the power to the ESP8266.  My new code resent all AT command to start up the server again and I can refresh my web browsers with no problems.

Testing the “busy s…” was also pretty easy.  I hit the browser refresh… waited 1 second… hit the “x” on the browser to about the request….. wait 1 second… tried to request the page again… The HW reset was toggled and the AT commands started the server.  I can then request the page again from the browser and everything looks ok.

With that, I think I can recover from most problems I have been seeing.

The code

Here is my code with all the additions so far.  Tired of copy pasting the entire code.  If you want it, save to your local drive from here:

You can open it in a text editor if you don’t have the Arduino IDE.