bash - merging big text files together -


i merging many files (~1 gig each) 1 file merged file incomplete. when concatenating b a, b gets concatenated somewhere in middle rather end. command running are:

for f in $x/*/y/*.fastq;      fullpath=`echo $(readlink -f $f)`     basename=`echo "${fullpath##*/}"`     pathname=`echo "${fullpath%/*}"`     name=`echo "$basename"|sed 's/-_-.*//'`      cat $f>>$x/z/${name}.fastq  done 

also, alternatively

names=$(cut -f 3 $b) names=$(echo "${names[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ') name in ${names[@]};     cat $x/*/y/${name}-_-*.fastq>$x/z/${name}.fastq done 

after inspect file, merged file has smaller size original , concatenated somewhere in middle.

thanks

try one:

#!/bin/bash shopt -s nullglob f in "$x"/*/y/*.fastq;     fullpath=$(readlink -f "$f")     basename=${fullpath##*/}     pathname=${fullpath%/*}     name=${basename%%-_-*}     dest=$x/z/$name.fastq     echo "$f >> $dest"     cat "$f" >> "$dest" done 

Comments

Popular posts from this blog

java - How to specify maven bin in eclipse maven plugin? -

single sign on - Logging into Plone site with credentials passed through HTTP -

php - Why does AJAX not process login form? -