bash - merging big text files together -
i merging many files (~1 gig each) 1 file merged file incomplete. when concatenating b a, b gets concatenated somewhere in middle rather end. command running are:
for f in $x/*/y/*.fastq; fullpath=`echo $(readlink -f $f)` basename=`echo "${fullpath##*/}"` pathname=`echo "${fullpath%/*}"` name=`echo "$basename"|sed 's/-_-.*//'` cat $f>>$x/z/${name}.fastq done also, alternatively
names=$(cut -f 3 $b) names=$(echo "${names[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ') name in ${names[@]}; cat $x/*/y/${name}-_-*.fastq>$x/z/${name}.fastq done after inspect file, merged file has smaller size original , concatenated somewhere in middle.
thanks
try one:
#!/bin/bash shopt -s nullglob f in "$x"/*/y/*.fastq; fullpath=$(readlink -f "$f") basename=${fullpath##*/} pathname=${fullpath%/*} name=${basename%%-_-*} dest=$x/z/$name.fastq echo "$f >> $dest" cat "$f" >> "$dest" done
Comments
Post a Comment