Website controlled Arduino Ethernet

[boomber]

Description:
There are many pages online offering to show you how to host a website from an Arduino Ethernet, and control the Arduino from that hosted website, but those mainly share the same method: sending a request by changing the website url address.
In this tutorial, I will be using the second available HTML method: sending a request by actually sending a DATA packet.


Video



Hardware:
Arduino Ethernet: http://www.jayconsystems.com/product_detail.php?prod_id=172
Ardumoto Shield V2: http://www.jayconsystems.com/product_detail.php?prod_id=114
LED wired to pins D6 and D7
DC motor using channel B of the Ardumoto Shield V2


Result
Here's a screenshot of what I get with the code in this tutorial.
Hosted by an Arduino! That's pretty neat, and this tutorial will get you to that!





Advantages or this method
- You can send any data type (including files), and are not limited to ASCII
- You can send any data length and are not limited by the url size
- Your data is Hidden, can be encrypted, and is difficult to hack (has to be used for applications using passwords)


Drawbacks or this method
- The server (here the Arduino) processing is more complex
- The processing depends on the web browser used, as different browsers send different request headers


Important: This tutorial is working for Firefox and will need to be adapted for any other browser. But I will explain how to modify it to get it working with your favorite browser!!


Code
If you want to get to business immediately, here is the whole code.
It is well commented but the whole thing might still be confusing, so I will explain every part more in detail in the rest of this tutorial

Code: Select all

#include <SPI.h>
#include "Ethernet.h"

// Ethernet library configuration
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };  //physical mac address
byte ip[] = { 192, 168, 1, 123 };                     // ip in lan
byte gateway[] = { 192, 168, 1, 1 };                  // internet access via router
EthernetServer server(80);                            //server port

// HMTL processing variables
String readString = "";             //string to get incoming data

char c;
char buffer[10];
int dataLength =0;

// Buffer containing processed hmtl data
char data[50];
char datamotor[5];
int  index=3;

// Arduino controls variables
  // Motor control Ardumoto B  PWM D5  Direction D8
   int motor_PINpwm = 5;
   int motor_PINdir = 8;
   int motor_VALpwm = 0;
   int motor_VALdir = LOW;
   
   int LEDpinlow = 6;     //simulated ground
   int LEDpin =    7;
   int LEDval=0;
   


// Setup function
void setup()
{
    Ethernet.begin(mac, ip, gateway);   //start Ethernet
    Serial.begin(9600);
   
    // Arduino controls initialization
    pinMode(motor_PINdir, OUTPUT);
    pinMode(LEDpinlow, OUTPUT);
    digitalWrite(LEDpinlow, LOW);
    pinMode(LEDpin, OUTPUT);
}


// Loop function
void loop()
{
  EthernetClient client = server.available();
  if (client)
  {
     while (client.connected())
     {
        while (client.available())  // Receive client data
        {
         
          Serial.print(".");
          c = client.read();   //read char by char HTTP request
          readString +=c;
          //Serial.print(c);          //output chars to serial port
         
          // If first request upon connexion, the 3 first characters will be "GET"
          // If "GET" is caught, skip the request info
          if( readString.equals("GET"))
          {
            Serial.println("");
            Serial.println("GET caught, skipping request and printing HTML");
            break;
          }
         
          // Otherwise, if the request contains data,
          // the first characters will be "POST"
          // We then skip the request header and this "if" becomes our main function
          if( readString.equals("POST"))
          {
            Serial.println("");
            Serial.println("POST caught, skipping header and acquiring DATA");
            // 320 is arbitrary. The actual length that has to be skipped depends on
            // several user settings ( browser, language, addons...)
            // the skipped length has not to be too long to skip relevant data
            // and not to short to waste computing time
            for(int i=0; i<320; i++)
            {
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
            //Searches for "Length: "
            while(c  != 'L')
            {             
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
            // Skip "Length: "
            for (int i=0; i<7; i++)
            {
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
           
            // Read the data package length
            readString="";
            c = client.read();
           
            while(c != '\n')
            {
              readString += c;
              //Serial.print(c);
              c = client.read();   
            }
           
            // convert data read from String to int
            readString.toCharArray(buffer, readString.length());
            dataLength = atoi(buffer);
            Serial.println("");
            Serial.print("dataLength: ");
            Serial.println(dataLength);
           
            // gets DATA
            client.read(); // skips additional newline
            client.read();
            for (int i=0; i<dataLength; i++)
            {
              data[i] = client.read();
            }
           
            Serial.println("");
            Serial.print("data: ");
            Serial.println(data);
           
            readString ="";
           
           
   // Handle data
            // Sort cases depending on the first 2 characters
           
            char tempbuff[3];
            tempbuff[0] = data[0];
            tempbuff[1] = data[1];
            switch (atoi(tempbuff)) //convert to int
            {
              case 1:

              Serial.println("In case 1");
             
              while(data[index] !='&')
              { datamotor[index-3] = data[index];
                index++; }
              motor_VALpwm = atoi(datamotor);
              if(motor_VALpwm > 255){motor_VALpwm = 255;}
              if(motor_VALpwm < 0){motor_VALpwm = 0;}
              Serial.println("PWM value: ");
              Serial.println(motor_VALpwm);
              index++;
              if( atoi(&data[index]) == 0)
              { motor_VALdir = LOW; }
              if( atoi(&data[index]) == 1)
              { motor_VALdir = HIGH; }
              Serial.println("PWM direction: ");
              Serial.println(motor_VALdir);
             
              digitalWrite(motor_PINdir, motor_VALdir);
              analogWrite(motor_PINpwm, motor_VALpwm);
             
              break;
             
             
             
              case 2:
              Serial.println("In case 2");
              if (LEDval == 0)
              {LEDval =1;}
              else
              {LEDval=0;}
              digitalWrite(LEDpin, LEDval);
              Serial.print("LED status: ");
              Serial.println(LEDval);
             
              break;
             
            }
            Serial.println("Out of switch");
           
          }       
        }
       
       
        // HTML CODE
        client.println("<!DOCTYPE html>");
        client.println("<html>");
          client.println("<body background=\"http://img256.imageshack.us/img256/5867/52577012.jpg\"</body>");
          client.println("<font color=white><h1 align=center>Jayconsystems Arduino Ethernet presentation</font></h1>");
          client.println("<P style=text-align:center><img src=\"http://img52.imageshack.us/img52/4299/logovgn.png\" width=600 height=150 /></P>");
          client.println("<hr />");
          client.println("<hr />");
            client.println("<form name=input method=post>");
                       
                String tempSTR = "<font color=white> Motor speed: <input type=text name=01 value=";
                tempSTR += motor_VALpwm;
                tempSTR += " maxlength=3 /><br />";
              client.println(tempSTR);
              client.println("<input type=submit name=0 value=\"DIRECTION 1\" />");
              client.println("<input type=submit name=1 value=\"DIRECTION 2\" />");
            client.println("</form> <br /> <br />");
           
            client.println("<dd><dd><dd>");
           
            if(LEDval == 1)
              {
                client.println("<img src=\"http://img688.imageshack.us/img688/4420/greencw.png\" width=20 height=20 />");
              }
              if(LEDval == 0)
              {
                client.println("<img src=\"http://img507.imageshack.us/img507/8505/blackko.png\" width=20 height=20 />");
              }
       
            client.println("<form name=input method=post>");
              client.println("<input type=hidden name=02 />");
              client.println("<input type=submit value=\"      LED      \"  />");
             
             
             
            client.println("</form> <br />");
           
          client.println("</body>");
        client.println("</html>");
       
        Serial.println("__________");
        delay(100);
       
        client.stop();
       
     }
  }



   
  // Reinitializing variables
  readString ="";          // Reinitialize String
  for (int i=0; i<10; i++)
    {buffer[i] = '\0';}
  for (int i=0; i<50; i++)
    {data[i] = '\0';}
  for (int i=0; i<5; i++)
    {datamotor[i] = '\0';}
  dataLength =0;
  index = 3;

}

Code detailed

Ethernet configuration

Code: Select all

#include <SPI.h>
#include "Ethernet.h"

Those are the basic include to get the Arduino Ethernet Library working

Code: Select all

// Ethernet library configuration
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };  //physical mac address
byte ip[] = { 192, 168, 1, 123 };                     // ip in lan
byte gateway[] = { 192, 168, 1, 1 };                  // internet access via router
EthernetServer server(80);                            //server port

The physical mac address is set by the code. Just make sure to change at least one of those values if you use 2 Arduinos on your network.
The IP in lan is also set by this code.
The gateway is the local address of my router
The EthernetServer server(80) means that a server is open on port 80 (standard port for HTTP)


Browser request message handler

Code: Select all

Serial.println("");
            Serial.println("POST caught, skipping header and acquiring DATA");
            // 320 is arbitrary. The actual length that has to be skipped depends on
            // several user settings ( browser, language, addons...)
            // the skipped length has not to be too long to skip relevant data
            // and not to short to waste computing time
            for(int i=0; i<320; i++)
            {
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
            //Searches for "Length: "
            while(c  != 'L')
            {             
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
            // Skip "Length: "
            for (int i=0; i<7; i++)
            {
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }

This is the part that depends on the browser and user.
You can uncomment the 3 lines labeled "// UNCOMMENT FOR DEBUG" to output on serial the browser-dependent header and request.

Then you will have to adapt that code on that info, by skipping irrelevant information and starting the data handling at specific detected characters.
If there's a high demand, I will work on a header analyzer that detects the browser and accordingly skips unrelevant data.


Caught Data management


Whenever a request is sent from the browser by clicking a button, one data value will be first detected.
For performance, I then organized data handler with switch/case.

Important: In my code, every case has to be a 2-numbers code. You can then with my code handle 99 different buttons.
(case 1 is actually case 01 but the 0 here doesn't matter)

I will just analyze the most complicated case of this code, the request controlling the motor:
The data sent by the browser will look like: data: "01=50&0=DIRECTION+1"

I will detail later that I gave the button "DIRECTION 1" the html name "0" and "DIRECTION 2" the html name "1"
The part that interests us here is "01=50&0".
It means: case 01 (motor) = 50 (speed value, ranging from 0 to 255) & 0 (direction 1).

A request demanding a 100% speed on direction 2 would then be: "01=255&1=DIRECTION+2"

Let's analyze the data handler then:

// I am there catching the speed value:
while(data[index] !='&')
{ datamotor[index-3] = data[index];
index++; }
// I am here converting the read buffered ASCII characters into the actual integer
motor_VALpwm = atoi(datamotor);
// Ensuring that limit values aren't exceeded
if(motor_VALpwm > 255){motor_VALpwm = 255;}
if(motor_VALpwm < 0){motor_VALpwm = 0;}
// Printing values
Serial.println("PWM value: ");
Serial.println(motor_VALpwm);
// Incrementing the data index to skip the "&" character
index++;
// Reading the next character. 0 means direction 1, 1 means direction 2
if( atoi(&data[index]) == 0)
{ motor_VALdir = LOW; }
if( atoi(&data[index]) == 1)
{ motor_VALdir = HIGH; }
Serial.println("PWM direction: ");
Serial.println(motor_VALdir);
// Updating the motor controls
digitalWrite(motor_PINdir, motor_VALdir);
analogWrite(motor_PINpwm, motor_VALpwm);
// Exiting the switch
break;



Website HTML construction

If you're still following me to that point, I got good news, the hardest part is done!!
The only thing left is the html output to refresh the website!


The goal of this tutorial is not to teach you html, there are plenty of great websites for that.
I will just explain the buttons:

Motor control:

//This is the "method" I was talking about at the beginning of this tutorial. The 2 different methods are "get" //and "post", and we are here using "post"
//Every user-input has to be a "form" content. You can put almost anything within the form.
//Our first form includes a user-input field and 2 buttons.

client.println("<form name=input method=post>");
// I give the form name "01". This can be whatever you want, but remember that the data analyzer is only //coded for 2 digits values and then the "0" is important here.
// I created a temp String to be able to print the current motor speed in the field, which is the
// "motor_VALpwm" variable
String tempSTR = "<font color=white> Motor speed: <input type=text name=01 value=";
tempSTR += motor_VALpwm;
tempSTR += " maxlength=3 /><br />";
client.println(tempSTR);
// We are still in the "form" and now I create the 2 buttons. The value is the text of the buttons. This can also
// be an image. The name of the buttons is what matters. Here 0 and 1, for 2 different cases
// (direction 1 and direction 2)
client.println("<input type=submit name=0 value=\"DIRECTION 1\" />");
client.println("<input type=submit name=1 value=\"DIRECTION 2\" />");
// And now we close the form
client.println("</form> <br /> <br />");



LED control:
This one is way easier.
We create one form with the name "02" (for case "02"), hide the user input field, and just put one button

client.println("<form name=input method=post>");
client.println("<input type=hidden name=02 />");
client.println("<input type=submit value=\" LED \" />");
client.println("</form> <br />");


Conclusion:

Alright, I guess that's it!
I admit that it's a pretty tricky one, there are plenty of ways to get lost between Arduino code and html, but I hope that I was of a great help!
Don't hesitate to ask questions here and share with us the great projects this tutorial helped you realize!!

_______________________

EDIT:
11/13/2012: Jonkapteijn points out that the header might have to be changed
http://www.jayconsystems.com/forum/viewtopic.php?f=10&t=90&p=917#p916

Try replacing:

Code: Select all

client.println("<body background=\"http://img256.imageshack.us/img256/5867/52577012.jpg\"</body>");

With:

Code: Select all

client.println("<body background=\"http://img256.imageshack.us/img256/5867/52577012.jpg\">");

[Iulian]

Hi!

Thank you for the helpful tutorial. I have two questions: 1. How can i modify the surse code in order to be able to connect from other browser (google chrome, safari by iphone). 2. Is it possible, to connect from another internet connection outside the local one? My internet connection type is PPPoE via router. I don't know very much about programming. I'm an electronist.

I would be glad if you could answer my questions.

Thank you very much.

Vasi

[boomber]

Hi!

For question 1, you need to uncomment the lines where it says
"Serial.print(c); // UNCOMMENT FOR DEBUG"

Then open the serial manager, send a request from the browser you want, and look at how the request looks like.

You'll then have to modify the whole "Browser request message handler" part to ignore irrelevant parts of that request and detect the data length, then capture the data. This is a bit troublesome and since you said you don't know much about programming, I'd advise you to stick with Firefox for now.

I had another idea since I made this tutorial: there might be another way, by just detecting the line where the data is, but I have to look at the different browser requests and adapt it from here. I'll maybe give it a shot in a few weeks since this is the major drawback of that tutorial/method for now.


For your question 2, you have to go to Port forwarding/Port triggering in your router admin panel, and reroute port 80 to your local Arduino IP address.
Then simply enter your IP address (if you don't know it: http://www.whatismyip.com/) in the browser, since this is an html request, it will be detected as port 80 by the router and redirected to your Arduino Ethernet

Here's a picture from my router to help you:


If redirecting port 80 interferes with other things you have on your network, it should be possible to define another port in Arduino, redirect that same port on your router, and access it from your browser with
"yourip:port" example: "123.456.7.8:99"

[Iulian]

Hi Boomber

thanks for your reply.

I managed to solve my second question but in a different way. I created a virtual server on my router because he has other settings than your router. Thank you for that!

I made the steps you suggested for solving my first question, but I dont know how to change the information in order to get a result. Here you have the requests: The first one is made with Firefox, the second with google chrome and the last with safari.

.P.O.S.T
POST caught, skipping header and acquiring DATA
/ HTTP/1.1
Host: 86.122.39.181
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ro-ro,ro;q=0.8,en-us;q=0.6,en-gb;q=0.4,en;q=0.2
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://86.122.39.181/
Content-Type: application/x-www-form-urlencoded
Content-Length:
dataLength: 3

data: 02=
In case 2
LED status: 1
Out of switch
__________
.P.O.S.T
POST caught, skipping header and acquiring DATA
/ HTTP/1.1
Host: 86.122.39.181
Connection: keep-alive
Content-Length: 3
Cache-Control: max-age=0
Origin: http://86.122.39.181
User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.47 Safari/536.11
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Referer: http://86.122.39.181/
Accept-Encoding: gzip,deflate,sdch
Accept-Language
dataLength: 0

data: =0.8,en-US;q=0.6,en;q=0.4
Out of switch
.c.e.p.t.-.C.h.a.r.s.e.t.:. .I.S.O.-.8.8.5.9.-.2.,.u.t.f.-.8.;.q.=.0...7.,.*.;.q.=.0...3.
.
.
.
.0.2.=__________
.G.E.T
GET caught, skipping request and printing HTML
__________

.P.O.S.T
POST caught, skipping header and acquiring DATA
/ HTTP/1.1
Host: 86.122.39.181
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3
Content-Length: 3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://86.122.39.181
Content-Type: application/x-www-form-urlencoded
Referer: http://86.122.39.181/
Accept-Language
dataLength: 0

data:
Out of switch
.c.e.p.t.-.E.n.c.o.d.i.n.g.:. .g.z.i.p.,. .d.e.f.l.a.t.e.
.
.C.o.n.n.e.c.t.i.o.n.:. .k.e.e.p.-.a.l.i.v.e.
.
.
.
.0.2.=__________


Could you please tell me what and how to change in the software in order to manage my problem?
Thanks a lot!

[boomber]

Well, see what I mean by "this is a troublesome one", everything is messed up on other ones than firefox
This comes from the fact that order and number of header lines are different.
I do in my code a dirty hard skip of irrelevant data, based on firefox.


And my code interferes with the headers reading.
To actually read the whole headers, do a simple loop with that:

Code: Select all

void loop()
{
  EthernetClient client = server.available();
  if (client)
  {
     while (client.connected())
     {
        while (client.available())  // Receive client data
        {
          c = client.read();         //read char by char HTTP request
          Serial.print(c);            //output chars to serial port
        }
     }
  }
}

Then you will have the whole headers, and from that you need to modify that whole part of the code:

Code: Select all

// Otherwise, if the request contains data,
          // the first characters will be "POST"
          // We then skip the request header and this "if" becomes our main function
          if( readString.equals("POST"))
          {
            Serial.println("");
            Serial.println("POST caught, skipping header and acquiring DATA");
            // 320 is arbitrary. The actual length that has to be skipped depends on
            // several user settings ( browser, language, addons...)
            // the skipped length has not to be too long to skip relevant data
            // and not to short to waste computing time
            for(int i=0; i<320; i++)
            {
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
            //Searches for "Length: "
            while(c  != 'L')
            {             
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
            // Skip "Length: "
            for (int i=0; i<7; i++)
            {
              c = client.read();
              //Serial.print(c); // UNCOMMENT FOR DEBUG
            }
           
            // Read the data package length
            readString="";
            c = client.read();
           
            while(c != '\n')
            {
              readString += c;
              //Serial.print(c);
              c = client.read();   
            }
           
            // convert data read from String to int
            readString.toCharArray(buffer, readString.length());
            dataLength = atoi(buffer);
            Serial.println("");
            Serial.print("dataLength: ");
            Serial.println(dataLength);
           
            // gets DATA
            client.read(); // skips additional newline
            client.read();
            for (int i=0; i<dataLength; i++)
            {
              data[i] = client.read();
            }
           
            Serial.println("");
            Serial.print("data: ");
            Serial.println(data);
           
            readString ="";

Try to understand what I did by comparing the code to the whole headers and you should be able to adapt it.
It would take me several hours to be more precise and modify the code, and I absolutely don't have the time this week.

But as I said, I'll surely work on that in the near future, and the more headers samples I have available, the cleaner I will be able to make the code. I understand you use an iPhone and I don't have one available, so please reshare the 3 different headers you get with the simple loop.

[TM21]

Hi,

Nice tutorial, now I can control my lights from a distance! (I'm working on a home automation project, all Arduino controlled).
Ok, for now it only works with firefox, so I'll be waiting for your 'other-browser-update'!

But: how about security? Do you already have something in common or perhaps some other suggestions?
Maybe even a statement like: not possible, there's allways a possibility some hacker will access your system...

I've knowledge of electronics and C, but not of networks, internet etc. unfortunally.

Anyway, thanks so far for your software, it really helps me ongoing.

Regards

[boomber]

I'm kind of like you on this topic, I have knowledge in electronics and C, and I had to learn Html basics for this tutorial.

As I mentioned in my tutorial, this method (POST) sends the data into the request.
The other method (GET) sends the data in the url, and anyone looking at your network traffic can see the url. (and thus any password/username used for authentication is immediately shown in clear)

With the method used in this tutorial, all data is "more" hidden.
I'm googling "post method security" to search about it. I then invite you to do the same to get your own understanding of how it works.


But about internet security, yes there's always a possibility for hackers to access your system. I'm not sure whether you can use https with an Arduino, as it requires more processing power. If security is really critical to your application then the best option would be to use an intermediate server.
Otherwise you should be fine from script-kiddies with the POST method (and who else than script kiddies would want to hack your lights? )

[TM21]

ok, the post method security opens a complete new world, let's dive into that!
First impression is that there are possibilities,but I've to study more on this.

On the other hand, maybe it's not that complicated anyway: I've got two Arduino Mega 2560's who do all the work (not only switch lights on and off, but also control my central heating system,fire alarm etc.)
The Arduino Ethernet is only to commicate to the 'outer world' via ethernet, and the 'inner world' (the other arduino's) via TWI.

So, if I put a password protection on the Arduino Mega's, no one can come into my real system.
Maybe the can hack into the Arduino ethernet, but they can't do there anything (there's no password stored in there).
In simple words: the mega's only accept commands via TWI when e.g. the first parameter sent is a valid password.

As typed all this stuff above, it sounds too easy ; am I overlooking something?

[boomber]

That makes sense and I don't really see a flow to that.
If the Ethernet is really just the relay and the other boards handle the password then the only way to access them would be to know the password.

[TM21]

That, in combination with the post method security, is the way it's gonna be.
Thanks for anticipating!