Sed advice needed

Piers Kittel (debian@biased.org) wrote on Thu, 31 May 2007 17:05:04 +0100:

>What's the best way to do this? I've been reading the man pages of
>sed, cut and awk but I can't quite figure out how to do this. Any
>ideas?

The sed man page is not very helpful I'm afraid. A pretty good manual is
here: .

Since you want to mask out various parts of your line you have to apply
sed to it various times, like this:

echo "your line" | sed (mask 1) | sed (mask 2) | sed (mask 3)

or, if your text comes from a file:

sed (mask 1) < yourFile | sed (mask 2) | sed (mask 3)

Hope this helps.

G. Heinrich

--

Here's one way:

# Put the sample lines into a file called inline.txt
for inline in `cat inline.txt | tr -d " "` #remove spaces for cut
do
# Remove quotes
Tmp=`echo $inline | tr -d "\""`
# Break into fields...
f1=`echo $Tmp | cut -f1 -d","`
f2=`echo $Tmp | cut -f2 -d","`
f3=`echo $Tmp | cut -f3 -d","`
f4=`echo $Tmp | cut -f4 -d","`
f5=`echo $Tmp | cut -f5 -d","`
# Get data past = char form the 3rd entry in 6th field
f6=`echo $Tmp | cut -f8 -d"," | cut -f2 -d"="`
echo "\"$f1\",\"$f2\",\"$f3\",\"$f4\",\"$f5\",\"$f6\""
done

Enjoy,
Larry
----- Original Message -----
From: "Piers Kittel"
To:
Sent: Thursday, May 31, 2007 12:05 PM
Subject: Sed advice needed

> Hello all,
>
> I need to delete some words out of a large file containing information
> about packets I'm analysing. I know I can use sed to do this, but
> haven't really used it before, so am a bit unsure of how to do it. Two
> example lines are as of below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54520, Time=1725612773, Mark"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54521, Time=1725616276"
>
> I need to convert the above to the below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","54520"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","54521"
>
> What's the best way to do this? I've been reading the man pages of sed,
> cut and awk but I can't quite figure out how to do this. Any ideas?
>
> Thanks very much for your time in advance!
>
> Regards - Piers
>
>
> --

On 31 May 2007, at 17:49, Tyler MacDonald wrote:

> Piers Kittel wrote:
>> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","Payloa
>> d t
>> ype=ITU-T H.261, SSRC=2008229573, Seq=54520, Time=1725612773, Mark"
>> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","Payloa
>> d t
>> ype=ITU-T H.261, SSRC=2008229573, Seq=54521, Time=1725616276"
>>
>> I need to convert the above to the below:
>>
>> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","54520"
>> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","54521"
>
> This *might* work, it depends on the rest of your dataset;
>
> sed -e 's/,"Payload.*Seq=//' -e 's/, Time=.*$/"/'
>
> Cheers,
> Tyler

That wasn't quite perfect, but I worked out a bit how the command
works and modified it - it now works like a dream! You're a
lifesaver :) I really appreciate this! I will learn exactly how the
command works - sed does seem quite powerful.

Thanks again!

Regards - Piers

--

On 2007-05-31 17:05 +0100, Piers Kittel wrote:

> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54520, Time=1725612773, Mark"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54521, Time=1725616276"
>
> I need to convert the above to the below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","54520"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","54521"

I don't know about "best" but one way to do it would be :

sed 's/Payload.*Seq=\([0-9]*\).*/\1"/'

--
André Majorel
Feeling lonely ? Report a bug in the Debian BTS and make new friends.

--

On 2007-05-31, Piers Kittel wrote:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54520, Time=1725612773, Mark"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54521, Time=1725616276"
>
> I need to convert the above to the below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","54520"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","54521"
>
> What's the best way to do this? I've been reading the man pages of
> sed, cut and awk but I can't quite figure out how to do this. Any
> ideas?
>

If I understand what you want, you need the fields 1-5 and 8 from a
comma-delimited record. At least, that's what cut will see - the
quotes around Payload...Mark will get ignored if you tell cut to use a
, as delimiter:

cut -d, -f1,2,3,4,5,8

Then you just need to get rid of the Seq= line, which sed will do
with:

sed s/Seq=//g

So put them together:

cut -d, -f1,2,3,4,5,8 input.file | sed s/Seq=//g

That looks almost right. I'm missing the quotes around the last field.
Two more sed commands fix this:

cut -d, -f1,2,3,4,5,8 input.file |
sed -e s/Seq=//g -e 's/, /,"/g' -e 's/$/"/g'

That seems to do the job with your example text.

HTH,

Tyler

--

Piers Kittel wrote:
> Hello all,
>
> I need to delete some words out of a large file containing information
> about packets I'm analysing. I know I can use sed to do this, but
> haven't really used it before, so am a bit unsure of how to do it. Two
> example lines are as of below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","Payload
> type=ITU-T H.261, SSRC=2008229573, Seq=54520, Time=1725612773, Mark"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","Payload
> type=ITU-T H.261, SSRC=2008229573, Seq=54521, Time=1725616276"
>
> I need to convert the above to the below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","54520"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","54521"
>
> What's the best way to do this? I've been reading the man pages of sed,
> cut and awk but I can't quite figure out how to do this. Any ideas?
>
> Thanks very much for your time in advance!
>
> Regards - Piers
>
>
> --To UNSUBSCRIBE, email to a
> subject of "unsubscribe". Trouble? Contact
>

To rephrase your question and data, so I'm sure I understand and you
know what I'm doing: You want to print items 1, 2, 3, 4, and 5, below,
as is, with only the Seq value from item 6, enclosed in double quotes.

1: "181",
2: "1324.014027",
3: "111.111.111.111",
4: "111.111.111.111",
5: "RTP",
6: "Payload type=ITU-T H.261, SSRC=2008229573, Seq=54520,
Time=1725612773, Mark"

If my description is correct, this will do the trick, assuming that all
the lines in the file have the same spacing and relative positioning:

sed 's/"Pay.*Seq=\(.*\), Time.*$/"\1"/'

What I mean by the above assumption would be, for example, that a Seg
number of 386 would look like "Seq=386" rather than "Seg= 386", or
other formatting differences.

The above sed substitution finds the part of the string starting with a
double quote followed by Pay, then any characters repeated up to the
portion with Seq=, followed by anything at all up to a comma space
followed by the word Time followed by anything up to the end of the
line. The portion between the \(...\) is remembered and is accessed in
the substitute by the \1. So, this part of the line:

"Pay....Seq=#####....Mark"

is replaced with:

"#####"

Where #### is the sequence number for each line.

--
Bob McGowan

On 05/31/07 11:05, Piers Kittel wrote:
> Hello all,
>
> I need to delete some words out of a large file containing information
> about packets I'm analysing. I know I can use sed to do this, but
> haven't really used it before, so am a bit unsure of how to do it. Two
> example lines are as of below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","Payload
> type=ITU-T H.261, SSRC=2008229573, Seq=54520, Time=1725612773, Mark"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","Payload
> type=ITU-T H.261, SSRC=2008229573, Seq=54521, Time=1725616276"
>
> I need to convert the above to the below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","54520"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","54521"
>
> What's the best way to do this? I've been reading the man pages of sed,
> cut and awk but I can't quite figure out how to do this. Any ideas?
>
> Thanks very much for your time in advance!

awk might be a better fit for this.

--
Ron Johnson, Jr.
Jefferson LA USA

Give a man a fish, and he eats for a day.
Hit him with a fish, and he goes away for good!

--

On 2007-05-31T17:05:04+0100, Piers Kittel wrote:
> I need to delete some words out of a large file containing
> information about packets I'm analysing. I know I can use sed to do
> this, but haven't really used it before, so am a bit unsure of how to
> do it. Two example lines are as of below:

This is a comma separated file (CSV), so your best bet is using an
utility that can parse that format and can handle the escape rules used.

Others already gave you suitable sed commands (assuming you want to
filter all files of the line), and here is the one that does it with
cut:

cut -d, -f1-5 $input > $ouput

/Allan

--

Piers Kittel wrote:
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54520, Time=1725612773, Mark"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","Payload t
> ype=ITU-T H.261, SSRC=2008229573, Seq=54521, Time=1725616276"
>
> I need to convert the above to the below:
>
> "181","1324.014027","111.111.111.111","111.111.111.111","RTP","54520"
> "185","1324.078941","111.111.111.111","111.111.111.111","RTP","54521"

This *might* work, it depends on the rest of your dataset;

sed -e 's/,"Payload.*Seq=//' -e 's/, Time=.*$/"/'

Cheers,
Tyler

--