76

Can we use shell variables in AWK like $VAR instead of $1, $2? For example:

UL=(AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW)

NUSR=`echo ${UL[*]}|awk -F, '{print NF}'`
echo $NUSR
echo ${UL[*]}|awk -F, '{print $NUSR}'

Actually am an oracle DBA we get lot of import requests. I'm trying to automate it using the script. The script will find out the users in the dump and prompt for the users to which dump needs to be loaded.

Suppose the dumps has two users AKHIL, SWATHI (there can be may users in the dump and i want to import more number of users). I want to import the dumps to new users AKHIL_NEW and SWATHI_NEW. So the input to be read some think like AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW.

First, I need to find the Number of users to be created, then I need to get new users i.e. AKHIL_NEW,SWATHI_NEW from the input we have given. So that I can connect to the database and create the new users and then import. I'm not copying the entire code: I just copied the code from where it accepts the input users.

UL=(AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW) ## it can be many users like     USER1:USER1_NEW,USER2_USER2_NEW,USER3:USER_NEW..

NUSR=`echo ${UL[*]}|awk -F, '{print NF}'` #finding  number of fields or users
y=1
while [ $y -le $NUSR ] ; do
    USER=`echo ${UL[*]}|awk -F, -v NUSR=$y  '{print $NUSR}' |awk -F: '{print $2}'` #getting     Users to created AKHIL_NEW and SWATHI_NEW and passing to SQLPLUS
    if [[ $USER = SCPO* ]]; then
        TBS=SCPODATA
    else
        if [[ $USER = WWF* ]]; then
            TBS=WWFDATA
        else
            if [[ $USER = STSC* ]]; then
                TBS=SCPODATA
            else
                if [[ $USER = CSM* ]]; then
                    TBS=CSMDATA
                else
                    if [[ $USER = TMM* ]]; then
                        TBS=TMDATA
                    else
                        if [[ $USER = IGP* ]]; then
                        TBS=IGPDATA
                        fi
                    fi
                fi
             fi
        fi
    fi

    sqlplus -s '/ as sysdba'  <<EOF   # CREATING the USERS in the database 
    CREATE USER $USER IDENTIFIED BY $USER  DEFAULT TABLESPACE $TBS TEMPORARY TABLESPACE TEMP QUOTA 0K on SYSTEM QUOTA UNLIMITED ON $TBS;

    GRANT
    CONNECT,
       CREATE TABLE,
       CREATE VIEW,
       CREATE SYNONYM,
       CREATE SEQUENCE,
       CREATE DATABASE LINK,
       RESOURCE,
       SELECT_CATALOG_ROLE
    to $USER;
    EOF 
    y=`expr $y + 1`
done

impdp sysem/manager DIRECTORY=DATA_PUMP DUMPFILE=imp.dp logfile=impdp.log SCHEMAS=AKHIL,SWATHI REMPA_SCHEMA=${UL[*]} 

In the last impdp command I need to get the original users in the dumps i.e AKHIL,SWATHI using the variables.

brandizzi
  • 26,083
  • 8
  • 103
  • 158
Akhil Chinnu
  • 771
  • 1
  • 5
  • 4
  • 1
    What was the question again? – Thor Apr 03 '13 at 12:07
  • 1
    The shell provides `elif [[ $USER = WWF* ]]; then` to avoid conditions marching off the RHS of the page and scads of `fi`'s. This script (currently) demonstrates why that is a valuable feature. – Jonathan Leffler May 11 '14 at 15:56
  • Possible duplicate of [How to use shell variables in awk script](http://stackoverflow.com/questions/19075671/how-to-use-shell-variables-in-awk-script) – tripleee Jun 01 '16 at 15:24
  • also see https://unix.stackexchange.com/questions/120788/pass-shell-variable-as-a-pattern-to-awk/120806#120806 - using ENVIRON array is preferable to -v due to escaped '\' issues – Dani_l Sep 25 '19 at 04:47

6 Answers6

129

Yes, you can use the shell variables inside awk. There are a bunch of ways of doing it, but my favorite is to define a variable with the -v flag:

$ echo | awk -v my_var=4 '{print "My var is " my_var}'
My var is 4

Just pass the environment variable as a parameter to the -v flag. For example, if you have this variable:

$ VAR=3
$ echo $VAR
3

Use it this way:

$ echo | awk -v env_var="$VAR" '{print "The value of VAR is " env_var}'
The value of VAR is 3

Of course, you can give the same name, but the $ will not be necessary:

$ echo | awk -v VAR="$VAR" '{print "The value of VAR is " VAR}'
The value of VAR is 3

A note about the $ in awk: unlike bash, Perl, PHP etc., it is not part of the variable's name but instead an operator.

brandizzi
  • 26,083
  • 8
  • 103
  • 158
  • 14
    The answer is NO! You can pass the value of a shell variable to an awk script just like you can pass the value of a shell variable to a C program but you cannot access a shell variable in an awk script any more than you could access a shell variable in a C program. Like C, awk is not shell. – Ed Morton Apr 03 '13 at 13:28
  • 4
    @EdMorton You can access environment variables in C. That's what they were originally designed for. – Zenexer Aug 23 '16 at 05:56
  • No, shell variables were not designed for access by C programs, they were designed for use in shell scripts. Normally I put the word `directly` in the comment to try to emphasize the point that while you can get at their values by calling an access function in C (`getenv`) or looking them up in an array in some awks (`ENVIRON`) you can't just **use** them in awk or C like you can in a shell script.. – Ed Morton Aug 23 '16 at 14:23
  • 1
    @EdMorton "The environment" existed before shells that used it came about. Shells were created in part as an easy way to manipulate the environment. See `man 7 environ` for a very brief overview. – Zenexer Aug 25 '16 at 03:08
  • @Zenexer are you seriously trying to twist this into a discussion about the difference between "shell variables" and "environment variables" and which came first and what "access" means? Not interested. If you think you can use shell variables inside awk or shell scripts, please go ahead and good luck with that. – Ed Morton Aug 25 '16 at 06:01
  • 1
    @EdMorton The goal is to get data from the shell to awk. Using the environment would be an easy way of passing string metadata to a child process at launch; a simple export statement enables this. When shell variables are exported, they become environment variables, but the difference between the two is negligible; even bash treats them identically: [1](http://git.savannah.gnu.org/cgit/bash.git/tree/variables.c#n119), [2](http://git.savannah.gnu.org/cgit/bash.git/tree/variables.h#n82) I'm not trying to point out a technicality, but rather that you're making things unnecessarily complicated. – Zenexer Aug 30 '16 at 21:54
  • 1
    @Zenexer The question was whether a shell variable can be used, and not accessed. In all honesty, the answer is quite useful to many - and does indeed solve the author's problem, so someone should edit the question so as to ensure the answer and question match. – Rads Dec 02 '20 at 01:48
  • 1
    Thank you, @brandizzi. EM is being a bit pedantic methinks. – Frobozz Nov 15 '22 at 07:49
30

Awk and Gawk provide the ENVIRON associative array that holds all exported environment variables. So in your awk script you can use ENVIRON["VarName"] to get the value of VarName, provided that VarName has been exported before running awk.

Note ENVIRON is a predefined awk variable NOT a shell environment variable.

Since I don't have enough reputation to comment on the other answers I have to include them here!

The earlier answer showing $ENVIRON is incorrect - that syntax would be expanded by the shell, and probably result in expanding to nothing.

Further earlier comments about C not being able to access environment variable is wrong. Contrary to what is said above, C (and C++) can access environment variables using the getenv("VarName") function. Many other languages provide similar access (e.g., Java: System.getenv(), Python: os.environ, Haskell System.Environment, ...). Note in all cases access to environment variables is read-only, you cannot change an environment variable in a program and get that value back to the calling script.

David Parks
  • 30,789
  • 47
  • 185
  • 328
graham hanson
  • 306
  • 3
  • 6
  • 3
    +1 but this is totally incorrect: `in all cases access to environment variables is read-only`. You can change the environment but only for the current program or programs launched by current program. That said your last line is correct that you can't change environment of current shell by an external program. Another note, shell variable is different from environment variable although for the shell user they appear similar and are accessed in the same way (at least in `sh` and similar shells). – akostadinov Jul 07 '16 at 13:12
  • Also see https://unix.stackexchange.com/questions/120788/pass-shell-variable-as-a-pattern-to-awk/120806#120806 – Dani_l Sep 25 '19 at 04:45
  • Couldn't get this to work, the variable seems never set. `TEST=xxx; echo | awk '{print "Test is " ENVIRON["TEST"]}'` prints only `Test is` – not2savvy Jan 16 '23 at 11:25
14

There are two ways to pass variables to awk: one way is defining the variable in a command line argument:

$ echo ${UL[*]}|awk -F, -v NUSR=$NUSR '{print $NUSR}'
SWATHI:SWATHI_NEW

Another way is converting the shell variable to an environment variable using export, and reading the environment variable from the ENVIRON array:

$ export NUSR
$ echo ${UL[*]}|awk -F, '{print $ENVIRON["NUSR"]}'
SWATHI:SWATHI_NEW

Update 2016: The OP has comma-separated data and wants to extract an item given its index. The index is in the shell variable NUSR. The value of NUSR is passed to awk, and awk's dollar operator extracts the item.

Note that it would be simpler to declare UL as an array of more than one element, and do the extraction in bash, and take awk out of the equation completely. This however uses 0-based indexing.

UL=(AKHIL:AKHIL_NEW SWATHI:SWATHI_NEW)
NUSR=1
echo ${UL[NUSR]} # prints SWATHI:SWATHI_NEW
Joni
  • 108,737
  • 14
  • 143
  • 193
9

There is another way, but it could cause immense confusion:

$ VarName="howdy" ; echo | awk '{print "Just saying '$VarName'"}'
Just saying howdy
$

So you are temporarily exiting the single quote environment (which would normally prevent the shell from interpreting '$') to interpret the variable and then going back into it. It has the virtue of being relatively brief.

Philip Kearns
  • 377
  • 4
  • 14
  • Does this create any potential problems? Seems to be the best way so far – Code42 Apr 30 '18 at 06:57
  • Well first and most important: It is obtuse. If you were to put it in a shell script it could make for confusing reading, and confusion can lead to mistakes. Second, if you were to put it in a more complicated expression you could find yourself escaping the quotes and $ potentially making the code even more unreadble. – Philip Kearns May 03 '18 at 15:28
  • I've used this although it is terrible and will lead to madness if you aren't FREQUENTLY testing your script with something like Bats to ensure quote hell hasn't dragged you down to the 9th level with Dante. – dragon788 Aug 08 '18 at 21:16
  • @dragon788 Exactly :) – Philip Kearns Aug 20 '18 at 16:18
2

Not sure if i understand your question.

But lets say we got a variable number=3 and we want to use it istead of $3, in awk we can do that with the following code

results="100 Mbits/sec 110 Mbits/sec 90 Mbits/sec"
number=3    
speed=$(echo $results | awk '{print '"\$${number}"'}')

so the speed variable will get the value 110.

Hope this helps.

Anoroah
  • 1,987
  • 2
  • 20
  • 31
-1

No. You can pass the value of a shell variable to an awk script just like you can pass the value of a shell variable to a C program but you cannot access a shell variable in an awk script any more than you could access a shell variable in a C program. Like C, awk is not shell. See question 24 in the comp.unix.shell FAQ at cfajohnson.com/shell/cus-faq-2.html#Q24.

One way to write your code would be:

UL="AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW"
NUSR=$(awk -F, -v ul="$UL" 'BEGIN{print gsub(FS,""); exit}')
echo "$NUSR"
echo "$UL" | awk -F, -v nusr="$NUSR" '{print $nusr}' # could have just done print $NF

but since your original starting point:

UL=(AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW)

was declaring UL as an array with just one entry, you might want to rethink whatever it is you're trying to do as you may have completely the wrong approach.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • 1
    You can access an environment variable from a C program. That's what they were designed for: http://man7.org/linux/man-pages/man3/getenv.3.html The environment is technically a feature of the OS; shells just happens to utilize it heavily and expose it to the user. The core environment interface is, in fact, only accessible to native (e.g. C) programs. This, of course, all assumes we're talking about POSIX. – Zenexer Aug 23 '16 at 05:59
  • Since you know that much I assume you also understand the point I was making and are now just playing the part of the language lawyer (FYI there's an excellent newsgroup comp.lang.c for doing that). Yes, of course you can call a function (getenv in C) or access an array (ENVIRON in awk) to get at the value of a shell variable in a C or awk but what you cannot do is use that shell variable directly in a C program or awk script as you would in a shell script. – Ed Morton Aug 23 '16 at 14:47
  • 5
    No, I don't understand the point you're trying to make. Do you mean to say that shell variables can't be accessed directly as variables (i.e., with `$` notation)? If so, I don't really think that was the point of the question; that's more of a technicality. As we've seen, yes, it is possible--and quite straightforward--to access the environment from both awk and C. – Zenexer Aug 25 '16 at 03:02