sed & awk

bromonkey · 2019-05-13 · Original thread

  man bash  
  man awk  
  man sed  
  man grep

You can also do most of this stuff with Perl one liners if that suits your fancy.

sed + awk - https://www.amazon.com/sed-awk-Dale-Dougherty/dp/1565922255

awk - https://ia802309.us.archive.org/25/items/pdfy-MgN0H1joIoDVoI...

general *nix text processing - https://www.tldp.org/LDP/abs/html/textproc.html

hudibras · 2014-06-29 · Original thread

The O'Reilly Sed & Awk book is a lot of fun, too.

http://shop.oreilly.com/product/9781565922259.do

nikcub · 2011-04-06 · Original thread

The best way to learn the UNIX tools is in the context of shell scripting and using the combination of tools to achieve a purpose (such as parsing log files, setting up shortcuts as part of your dev toolchain)

1. Understand the philosophy behind UNIX. ie. everything is a file and it is small apps that are very good at a single task that can be combined through pipes etc. to process larger tasks (this has lost its way somewhat in Linux, but still holds true)

2. Look at your computer usage through development, sysadmin, etc. and find parts that you want to automate, and then set out to write scripts to complete these tasks. for eg. in each one of my projects I have a script called start.sh which has a bunch of tasks implemented (eg. backup, serve, dns (to update dyndns for callbacks), push, diff, deploy, stage (run remote commands) etc.). Remember that git is itself a set of shell scripts, as are most server management commands. Take a peak inside each of these to learn how they are implemented.

3. Once you have a good understanding of what you want to implement, just go for it. Learn along the way by using Bash scripting, regular expressions, an understanding of UNIX/Linux, sed, awk, yacc/bison (to analyze code and extract info). You will end up building your own collection of shell scripts, environment variables, .vimrc, ssh, curl (you can do almost any API request and auth with curl and a shell script - automate tweets, RSS feeds) etc. (I have been meaning to publish my own, the most recent one I wrote greps my code for TODO's and pushes them to a simple webapp where I can view them, sort them, etc. along side my 'personal' todo list)

4. You need two sets of resources, one is for learning, the other is for reference which you keep handy. Here are my own recommendations:

Learning:

The learning style book comes down a lot to personal preference. You can go wrong, so get a feel for each topic by reading online tutorials and then scan the TOC and sample chapters of books that look good. A subscription to O'Reilly Safari comes in handy. There are now also a lot of screencasts online, try searching YouTube for the topic with 'screencast' or 'tutorial', once you find a good screencast publisher look at the rest of his/her videos.

You can also get the O'Reilly books on special in bundles sometimes, check their website

There is no book that really introduces UNIX and then covers most of these topics as I described. ie. 'UNIX for Developers'.

* UNIX:

- FreeBSD handbook: http://www.freebsd.org/doc/handbook/

- Linux Documentation Project: http://tldp.org/LDP/

- Linux Command Line and Shell Scripting Bible: http://www.amazon.com/Linux-Command-Shell-Scripting-Bible/dp...

- UNIX In a Nutshell (I haven't read this in a while but it is on my shelf): http://oreilly.com/catalog/9781565924277

- Learning UNIX (also O'Reilly): http://www.amazon.com/Learning-UNIX-Operating-System-Fifth/d...

* Bash:

- http://linuxcommand.org/

- http://bash.cyberciti.biz/guide/Main_Page (better, and v good)

- Learning the Bash shell: http://oreilly.com/catalog/9780596009656/

- Advanced Bash Scripting (online): http://tldp.org/LDP/abs/html/

- YouTube playlist: http://www.youtube.com/view_play_list?p=2284887FAE36E6D8

* Regular Expressions:

- Mastering Regular Expressions (this is such a great book. I didn't like regexp until I picked this up eons ago): http://oreilly.com/catalog/9781565922570

* sed/awk/grep etc.

- sed & awk (O'Reilly): http://oreilly.com/catalog/9781565922259

* Vim/Vi:

- vimcasts (screencasts): http://vimcasts.org

- more screencasts (these are great): http://vimeo.com/user1690209/videos

- learning vi: http://www.amazon.com/dp/1565924266

Reference:

On my browser toolbar, I have a folder called '_ref', it has around 70+ bookmarks (online documentation, cheatsheats) in it each titled what the link is a reference for (ie. 'vi', 'bash' etc.). I will publish it online, and will post the links here. I have collected the links over years.

ISBN: 9781565922259