TCL EXPECT TUTORIAL

Except :

Expect is a tool primarily for automating interactive applications such as telnet, ftp, password, , rlogin, etc. Expect really makes this stuff trivial. Expect is also useful for testing these same applications.

Pros :

Expect can  run at regular intervals through the use of cron to encapsulate system administration tasks. This works because Expect merely uses system administration tools already located on the host computer. No extra tools need to be learned. If the programmer has already learned Tcl, then migrating to Expect is a relatively easy transition. The same programming structures and syntax exist, but with additional features built in.

There is large support in the industry for using Expect for many in-house administration tasks. It is widely used by companies such as Silicon Graphics, IBM, HP, Sun, Xerox,, Tektronix, AT&T, to run in-house automated testing for development projects, file transfers, account administration, and network testing. 

As a TCL extension, expect uses the same syntax, which is more regular and uniform than languages such as bash,Perl. Expect goes to great lengths to abstract away the differences between terminal behaviour on  various platforms. Other programs similar to expect are usually lacking in this regard. Because it is a TCL extension, the full facilities of TCL are available for use in expect scripts.

Cons

Expect cannot automate GUI based tools. This is generally only a problem on Windows where for many tasks a GUI based interface is the only option. In these situations tools like AutoHotkey, AutoIt, or Winbatch can be used instead.

---

Get Started With Expect 

The three commands send, expect, and spawn are the building power of Expect. The send command sends strings to a process, the expect command waits for strings from a process, and the spawn command starts a process. 

---

The send Command

The send command takes a string as an argument and sends it to a process. For example: 

send "hello world" 

This sends the string "hello world" (without the quotes). If Expect is already interacting with a program, the string will be sent to that program. But initially, send will send to the standard output. Here is what happens when I type this to the Expect interpreter interactively: 

% expect

expect1.1>send "hello world"

hello worldexpect1.2>exit

%

The send command does not format the string in any way, so after it is printed the next Expect prompt gets appended to it without any space. To make the prompt appear on a different line, put a newline character at the end of the string. A newline is represented by "\n".

The exit command gets you out of the Expect interpreter. 

expect1.1>send "hello world\n"

hello world

expect1.2>exit

%

If these commands are stored in a file, speak, the script can be executed from the UNIX command line: 

% expect speak

hello world

To execute the file as just "speak" rather than "expect speak", insert the line "#!./expect -f" and do "chmod +x speak" . The name of the interpreter must appear after the characters #! in the first line. The ./expect is the path where Expect is to be found; in this case, it is in the current working directory. 

% cat speak

#!./expect -f

send "hello world\n" 

% 

% chmod +x speak

% speak

hello world

---

The expect Command

The expect command waits for a response, usually from a process. expect can wait for a specific string or any string that matches a given pattern. Like send, the expect command initially waits for characters from the keyboard. To see how see how the expect command works, create a a file response.exp that reads: 

#!./expect -f 

expect "hi\n"

send "hello there!\n"

When I make response.exp executable and run it, the interaction looks like this: 

% chmod +x response.exp

% response.exp

hi

hello there!

If you get an error that goes like couldn't read file " ": No such file or directory, it may be because there are non-printable characters in your file. This is true if you do cut-and-paste from Netscape to your file. To solve this problem, try deleting trailing spaces at the end of each command line (even if there seems to be nothing there) in the script and follow the above steps again. 

What Happens When Input Does Not Match

If expect reads characters that do not match the expected string, it continues waiting for more characters. If I had type hello instead of hi followed by a return, expect would continue to wait for "hi\n". Finding unexpected data in the input does not bother expect. It keeps looking until it finds something that matches. If no input is given, expect command eventually times out and returns. By default, after 10 seconds expect gives up waiting for input that matches the pattern. This default value can be changed by setting the variable timeout using the Tcl set command. For example, the following command sets the timeout to 60 seconds. 

set timeout 60 

A timeout of -1 signifies that expect should wait forever and a timeout of 0 indicates that expect should not wait at all. 

Anchoring

To prevent expect from matching unexpected data, expect patterns can include regular expressions. The caret ^ is a special character that only matches the beginning of the input; it cannot skip over characters to find a valid match. For example, the pather ^hi matches if I enter "hiccup" but not if I enter "sushi" . The dollar sign ($) is another special character. It matches the end of the data. The pattern hi$ matches if I enter "sushi" but not if I enter "hiccup". And the pattern ^hi$ matches neither "sushi" nor "hiccup". It matches "hi" and nothing else. 

Patterns that use ^ or $ are said to be anchored. When patterns are not anchored, patterns match beginning at the earliest possible position in the string. For more techniques on pattern matching, I suggest you buy the the book, Exploring Expect as well as Tcl and The Tk Toolkit. 

Pattern-Action Pairs

Expect also allows association between a command and a pattern. The association is made by listing the action (also known as command) immediately after the pattern in the expect command itself. Here is an example of pattern-action pairs: 

     expect "hi"  { send "You said hi\n" } \

    "hello"   { send "Hello yourself\n" } \

    "bye"     { send "Good-bye cruel world\n" }

This command looks for "hi", "hello", and "bye". If any of the three 

patterns are found, the action immediately following it gets executed. 

If there is no match and the default timeout expires, expect stops 

waiting and execution continues with the next command in the script. 

---

The spawn Command

The spawn command starts another program. The first argument of the spawn command is the name of a program to start. The remaining arguments are passed to the program. For example: 

spawn ftp ftp.uu.net 

This command spawns an ftp process and ftp.uu.net is the argument to the ftp process. 

---

Putting It All Together

#A Simple Example

#!/usr/local/bin/tclsh

#!/usr/bin/expect

package require Expect

#for {set i 1} {$i < 33} {incr i} {

spawn ssh 10.37.64.108

expect "Please Enter Login Name:"

send "user_name\r"

expect "Please Enter Password:"

send "password\r"

expect ".*>"

send "exit\r"

#expect ".*$"

#}

# The example script below telnets to the provided IP address and port number, executes a single command, namely a unix command “ls”, and then closes the telnet session.

#!/usr/bin/expect

set timeout 1

set ip [lindex $argv 0]

set port [lindex $argv 1]

set username [lindex $argv 2]

set password [lindex $argv 3]

spawn telnet $ip $port

expect “‘^]’.”

send – – “\r”

expect “username:” {

    send – – “$username\r”

    expect “password:”

    send – – “$password\r”

}

expect “$”

send – – “ls\r”

expect “$”

sleep 2

send “35\r”

expect “telnet>”

send – – “quit\r”

expect eof

The command to execute the script:

expect example <ip> <port> <username> <password>

More explanation on each line:

To begin with, you need to check if expect utility is installed on your machine. If not, install it with sudo apt-get install expect.

Next step is to check where exactly it is located on you machine. Run the command where is expect. The command will return a path such as “expect: 

/usr/bin/expect”.  

Create a new file, the extension does not matter. Let’s say we have a script file named example.  

Open the file for editing and the first line you need to add is

#!/usr/bin/expect

or use another path if you got another one in step 2.

Next, specify the input arguments to the script:

set ip [lindex $argv 0]

set port [lindex $argv 1]

set username [lindex $argv 2]

set password [lindex $argv 3]

You can also optionally add the default timeout to be 1 second:

set timeout 1

Start a telnet session towards the given IP address and port number:

spawn telnet $ip $port

When a telnet session is successfully started you usually get a message that looks something like that:

Trying 192.168.1.1…

Connected to 192.168.1.1.

Escape character is ‘^]’.

With Expect you need always to specify what you expect to see as output after executing a command, so in this example we expect the line to contain the 

escape symbol:

expect “‘^]’.”

\r is a symbol for Enter:

send — “\r”

Here is an interesting part. My remote machine can either trigger a user authentication or can skip it depending on when the last login took place. So I used 

a simple “if statement” to determine whether the script needed to provide the credentials for authentication or the “ls” command could be executed directly.

expect “username:” {

   send — “$username\r”

   expect “password:”

   send — “$password\r”

}

expect “$”

send — “ls\r”

expect “$”

Optionally you can suspend the script for example for 2 seconds before existing if the default timeout of 1 second is too short:

sleep 2

Finally the script is supposed to terminate the telnet session. For this it sends the command CTRL-] as octal  digits and then the quit command as usual. The 

ASCII codes for other common commands can be found here.

send “35\r

”

expect “telnet>”

send — “quit\r”

expect eof

#A simple example is a script that automates a telnet session:

Assume $remote_server, $my_user_id, $my_password, and $my_command were read in earlier  in the script.

Open a telnet session to a remote server, and wait for a username prompt.

spawn telnet $remote_server

expect "username:"

# Send the username, and then wait for a password prompt.

send "$my_user_id\r"

expect "password:"

# Send the password, and then wait for a shell prompt.

send "$my_password\r"

expect "%"

# Send the prebuilt command, and then wait for another shell prompt.

send "$my_command\r"

expect "%"

# Capture the results of the command into a variable. This can be displayed, or written to disk.

set results $expect_out(buffer)

# Exit the telnet session, and wait for a special end-of-file character.

send "exit\r"

expect eof

Another example is a script that automates ftp:

Set timeout parameter to a proper value. For example, the file size is indeed big and the network speed is really one problem,  you'd better set this parameter a value.

set timeout -1

# Open an ftp session to a remote server, and wait for a username prompt.

spawn ftp $remote_server

expect "username:"

# Send the username, and then wait for a password prompt.

send "$my_user_id\r"

expect "password:"

# Send the password, and then wait for an ftp prompt.

send "$my_password\r"

expect "ftp>"

# Switch to binary mode, and then wait for an ftp prompt.

send "bin\r"

expect "ftp>"

# Turn off prompting.

send "prompt\r"

expect "ftp>"

# Get all the files

send "mget *\r"

expect "ftp>"

# Exit the ftp session, and wait for a special end-of-file character.

send "bye\r"

expect eof

Another example of automated ssh login in user machine

Timeout is a predefined variable in expect which by default is set to 10 sec ,spawn_id is another default variable in expect. It is good practice to close spawn_id handle created by spawn command

set timeout 60 

spawn ssh $user@machine

while {1} {

  expect {

    eof                          {break}

    "The authenticity of host"   {send "yes\r"}

    "password:"                  {send "$password\r"}

    "*\]"                        {send "exit\r"}

  }

}

wait 

close $spawn_id

#######

expect_out(0,start) Index of the first character of the string that matched the entire expression

expect_out(0,end) Index of the last character of the string that matched the entire expression

expect_out(0,string) String that matched the entire expression

expect_out(1..9,start) Index of the first character of the string that matched the pattern enclosed in the 1st - 9th set of parentheses

expect_out(1..9,end) Index of the last character of the string that matched the pattern enclosed in the 1st - 9th set of parentheses

expect_out(1..9,string) String that matched the pattern enclosed in the 1st - 9th set of parentheses

expect_out(buffer) Entire contents of the buffer when a match was found

expect_out(spawn_id) Spawn id of the process which produced the matching pattern

Expect

Expect is a Tcl/Tk Extension

• Expect is a tool for automating interactive applications such as telnet,

ftp, passwd, fsck, rlogin, tip, etc. Expect really makes this stuff trivial.

• When to use it – Start an external application and simulate user interaction

Can start telnet to a router

Can send commands

Can capture router outputs

Expect commands:

Spawn – Spawn a new process (telnet,sh etc…)

• Expect – wait for the specified patterns

• Send – Send specified string to the spawned process

• Close – Terminate the process

Expect reserved variables:

timeout – specify the timeout (in seconds) for an expect command, default is 10

seconds.

• spawn_id – Unique id assigned to each process created by spawn command

. expect_out(buffer) = all characters up to matched pattern.

• expect_out(0,string) = characters that matched “pattern”.

• expect_out(,string) = characters matching subpattern .

Given the character stream “abcdefgh\n”, the variable 

contents would be:

expect "b(.*)e([a-z])g":

expect4.7> echo $expect_out(buffer)

abcdefg

expect4.8> echo $expect_out(0,string)

bcdefg

expect4.9> echo $expect_out(1,string)

cd

expect4.10> echo $expect_out(2,string)