副标题#e#
环境:
ubuntu 14.04
内存 6G
bwa 0.7.12
结论:
建立索引大概4500秒左右
节点2运行:
hadoop@Mcnode2:~/cloud/adam/xubo/data/test20160422$ cp ../test20160310/GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna . hadoop@Mcnode2:~/cloud/adam/xubo/data/test20160422$ ls GCA_000001405.15_GRCh38_full_analysis_set.fna hadoop@Mcnode2:~/cloud/adam/xubo/data/test20160422$ bwa index GCA_000001405.15_GRCh38_full_analysis_set.fna [bwa_index] Pack FASTA... 34.05 sec [bwa_index] Construct BWT for the packed sequence... [BWTIncCreate] textLength=6418915856,availableWord=463658232 [BWTIncConstructFromPacked] 10 iterations done. 100000000 characters processed. [BWTIncConstructFromPacked] 20 iterations done. 200000000 characters processed. [BWTIncConstructFromPacked] 30 iterations done. 300000000 characters processed. [BWTIncConstructFromPacked] 40 iterations done. 400000000 characters processed. [BWTIncConstructFromPacked] 50 iterations done. 500000000 characters processed. [BWTIncConstructFromPacked] 60 iterations done. 600000000 characters processed. [BWTIncConstructFromPacked] 70 iterations done. 700000000 characters processed. [BWTIncConstructFromPacked] 80 iterations done. 800000000 characters processed. [BWTIncConstructFromPacked] 90 iterations done. 900000000 characters processed. [BWTIncConstructFromPacked] 100 iterations done. 1000000000 characters processed. [BWTIncConstructFromPacked] 110 iterations done. 1100000000 characters processed. [BWTIncConstructFromPacked] 120 iterations done. 1200000000 characters processed. [BWTIncConstructFromPacked] 130 iterations done. 1300000000 characters processed. [BWTIncConstructFromPacked] 140 iterations done. 1400000000 characters processed. [BWTIncConstructFromPacked] 150 iterations done. 1500000000 characters processed. [BWTIncConstructFromPacked] 160 iterations done. 1600000000 characters processed. [BWTIncConstructFromPacked] 170 iterations done. 1700000000 characters processed. [BWTIncConstructFromPacked] 180 iterations done. 1800000000 characters processed. [BWTIncConstructFromPacked] 190 iterations done. 1900000000 characters processed. [BWTIncConstructFromPacked] 200 iterations done. 2000000000 characters processed. [BWTIncConstructFromPacked] 210 iterations done. 2100000000 characters processed. [BWTIncConstructFromPacked] 220 iterations done. 2200000000 characters processed. [BWTIncConstructFromPacked] 230 iterations done. 2300000000 characters processed. [BWTIncConstructFromPacked] 240 iterations done. 2400000000 characters processed. [BWTIncConstructFromPacked] 250 iterations done. 2500000000 characters processed. [BWTIncConstructFromPacked] 260 iterations done. 2600000000 characters processed. [BWTIncConstructFromPacked] 270 iterations done. 2700000000 characters processed. [BWTIncConstructFromPacked] 280 iterations done. 2800000000 characters processed. [BWTIncConstructFromPacked] 290 iterations done. 2900000000 characters processed. [BWTIncConstructFromPacked] 300 iterations done. 3000000000 characters processed. [BWTIncConstructFromPacked] 310 iterations done. 3100000000 characters processed. [BWTIncConstructFromPacked] 320 iterations done. 3200000000 characters processed. [BWTIncConstructFromPacked] 330 iterations done. 3300000000 characters processed. [BWTIncConstructFromPacked] 340 iterations done. 3400000000 characters processed. [BWTIncConstructFromPacked] 350 iterations done. 3500000000 characters processed. [BWTIncConstructFromPacked] 360 iterations done. 3600000000 characters processed. [BWTIncConstructFromPacked] 370 iterations done. 3700000000 characters processed. [BWTIncConstructFromPacked] 380 iterations done. 3800000000 characters processed. [BWTIncConstructFromPacked] 390 iterations done. 3900000000 characters processed. [BWTIncConstructFromPacked] 400 iterations done. 4000000000 characters processed. [BWTIncConstructFromPacked] 410 iterations done. 4100000000 characters processed. [BWTIncConstructFromPacked] 420 iterations done. 4200000000 characters processed. [BWTIncConstructFromPacked] 430 iterations done. 4300000000 characters processed. [BWTIncConstructFromPacked] 440 iterations done. 4400000000 characters processed. [BWTIncConstructFromPacked] 450 iterations done. 4500000000 characters processed. [BWTIncConstructFromPacked] 460 iterations done. 4600000000 characters processed. [BWTIncConstructFromPacked] 470 iterations done. 4700000000 characters processed. [BWTIncConstructFromPacked] 480 iterations done. 4800000000 characters processed. [BWTIncConstructFromPacked] 490 iterations done. 4900000000 characters processed. [BWTIncConstructFromPacked] 500 iterations done. 5000000000 characters processed. [BWTIncConstructFromPacked] 510 iterations done. 5100000000 characters processed. [BWTIncConstructFromPacked] 520 iterations done. 5200000000 characters processed. [BWTIncConstructFromPacked] 530 iterations done. 5300000000 characters processed. [BWTIncConstructFromPacked] 540 iterations done. 5400000000 characters processed. [BWTIncConstructFromPacked] 550 iterations done. 5500000000 characters processed. [BWTIncConstructFromPacked] 560 iterations done. 5600000000 characters processed. [BWTIncConstructFromPacked] 570 iterations done. 5700000000 characters processed. [BWTIncConstructFromPacked] 580 iterations done. 5798188880 characters processed. [BWTIncConstructFromPacked] 590 iterations done. 5886472096 characters processed. [BWTIncConstructFromPacked] 600 iterations done. 5964934432 characters processed. [BWTIncConstructFromPacked] 610 iterations done. 6034667936 characters processed. [BWTIncConstructFromPacked] 620 iterations done. 6096643264 characters processed. [BWTIncConstructFromPacked] 630 iterations done. 6151723072 characters processed. [BWTIncConstructFromPacked] 640 iterations done. 6200674128 characters processed. [BWTIncConstructFromPacked] 650 iterations done. 6244177920 characters processed. [BWTIncConstructFromPacked] 660 iterations done. 6282840176 characters processed. [BWTIncConstructFromPacked] 670 iterations done. 6317199264 characters processed. [BWTIncConstructFromPacked] 680 iterations done. 6347733664 characters processed. [BWTIncConstructFromPacked] 690 iterations done. 6374868704 characters processed. [BWTIncConstructFromPacked] 700 iterations done. 6398982368 characters processed. [BWTIncConstructFromPacked] 710 iterations done. 6418915856 characters processed. [bwt_gen] Finished constructing BWT in 710 iterations. [bwa_index] 3102.10 seconds elapse. [bwa_index] Update BWT... 23.50 sec [bwa_index] Pack forward-only FASTA... 21.21 sec [bwa_index] Construct SA from BWT and Occ... 1087.39 sec [main] Version: 0.7.12-r1039 [main] CMD: bwa index GCA_000001405.15_GRCh38_full_analysis_set.fna [main] Real time: 4444.462 sec; CPU: 4268.248 sec
#p#副标题#e##p#分页标题#e#
hadoop@Mcnode2:~/cloud/adam/xubo/data/test20160422$ ll -h total 8.3G drwxrwxr-x 2 hadoop hadoop 4.0K 4月 22 16:36 ./ drwxrwxr-x 4 hadoop hadoop 4.0K 4月 22 15:20 ../ -rw------- 1 hadoop hadoop 3.1G 4月 22 15:22 GCA_000001405.15_GRCh38_full_analysis_set.fna -rw-rw-r-- 1 hadoop hadoop 20K 4月 22 16:18 GCA_000001405.15_GRCh38_full_analysis_set.fna.amb -rw-rw-r-- 1 hadoop hadoop 72K 4月 22 16:18 GCA_000001405.15_GRCh38_full_analysis_set.fna.ann -rw-rw-r-- 1 hadoop hadoop 3.0G 4月 22 16:17 GCA_000001405.15_GRCh38_full_analysis_set.fna.bwt -rw-rw-r-- 1 hadoop hadoop 766M 4月 22 16:18 GCA_000001405.15_GRCh38_full_analysis_set.fna.pac -rw-rw-r-- 1 hadoop hadoop 1.5G 4月 22 16:37 GCA_000001405.15_GRCh38_full_analysis_set.fna.sa
节点3运行:
hadoop@Mcnode3:~/cloud/adam/xubo/data/test20160422$ cp ../test20160310/GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna . hadoop@Mcnode3:~/cloud/adam/xubo/data/test20160422$ free -m total used free shared buffers cached Mem: 5960 5851 109 0 149 4482 -/+ buffers/cache: 1218 4742 Swap: 6133 314 5819 hadoop@Mcnode3:~/cloud/adam/xubo/data/test20160422$ bwa index GCA_000001405.15_GRCh38_full_analysis_set.fna [bwa_index] Pack FASTA... 33.06 sec [bwa_index] Construct BWT for the packed sequence... [BWTIncCreate] textLength=6418915856,availableWord=463658232 [BWTIncConstructFromPacked] 10 iterations done. 100000000 characters processed. [BWTIncConstructFromPacked] 20 iterations done. 200000000 characters processed. [BWTIncConstructFromPacked] 30 iterations done. 300000000 characters processed. [BWTIncConstructFromPacked] 40 iterations done. 400000000 characters processed. [BWTIncConstructFromPacked] 50 iterations done. 500000000 characters processed. [BWTIncConstructFromPacked] 60 iterations done. 600000000 characters processed. [BWTIncConstructFromPacked] 70 iterations done. 700000000 characters processed. [BWTIncConstructFromPacked] 80 iterations done. 800000000 characters processed. [BWTIncConstructFromPacked] 90 iterations done. 900000000 characters processed. [BWTIncConstructFromPacked] 100 iterations done. 1000000000 characters processed. [BWTIncConstructFromPacked] 110 iterations done. 1100000000 characters processed. [BWTIncConstructFromPacked] 120 iterations done. 1200000000 characters processed. [BWTIncConstructFromPacked] 130 iterations done. 1300000000 characters processed. [BWTIncConstructFromPacked] 140 iterations done. 1400000000 characters processed. [BWTIncConstructFromPacked] 150 iterations done. 1500000000 characters processed. [BWTIncConstructFromPacked] 160 iterations done. 1600000000 characters processed. [BWTIncConstructFromPacked] 170 iterations done. 1700000000 characters processed. [BWTIncConstructFromPacked] 180 iterations done. 1800000000 characters processed. [BWTIncConstructFromPacked] 190 iterations done. 1900000000 characters processed. [BWTIncConstructFromPacked] 200 iterations done. 2000000000 characters processed. [BWTIncConstructFromPacked] 210 iterations done. 2100000000 characters processed. [BWTIncConstructFromPacked] 220 iterations done. 2200000000 characters processed. [BWTIncConstructFromPacked] 230 iterations done. 2300000000 characters processed. [BWTIncConstructFromPacked] 240 iterations done. 2400000000 characters processed. [BWTIncConstructFromPacked] 250 iterations done. 2500000000 characters processed. [BWTIncConstructFromPacked] 260 iterations done. 2600000000 characters processed. [BWTIncConstructFromPacked] 270 iterations done. 2700000000 characters processed. [BWTIncConstructFromPacked] 280 iterations done. 2800000000 characters processed. [BWTIncConstructFromPacked] 290 iterations done. 2900000000 characters processed. [BWTIncConstructFromPacked] 300 iterations done. 3000000000 characters processed. [BWTIncConstructFromPacked] 310 iterations done. 3100000000 characters processed. [BWTIncConstructFromPacked] 320 iterations done. 3200000000 characters processed. [BWTIncConstructFromPacked] 330 iterations done. 3300000000 characters processed. [BWTIncConstructFromPacked] 340 iterations done. 3400000000 characters processed. [BWTIncConstructFromPacked] 350 iterations done. 3500000000 characters processed. [BWTIncConstructFromPacked] 360 iterations done. 3600000000 characters processed. [BWTIncConstructFromPacked] 370 iterations done. 3700000000 characters processed. [BWTIncConstructFromPacked] 380 iterations done. 3800000000 characters processed. [BWTIncConstructFromPacked] 390 iterations done. 3900000000 characters processed. [BWTIncConstructFromPacked] 400 iterations done. 4000000000 characters processed. [BWTIncConstructFromPacked] 410 iterations done. 4100000000 characters processed. [BWTIncConstructFromPacked] 420 iterations done. 4200000000 characters processed. [BWTIncConstructFromPacked] 430 iterations done. 4300000000 characters processed. [BWTIncConstructFromPacked] 440 iterations done. 4400000000 characters processed. [BWTIncConstructFromPacked] 450 iterations done. 4500000000 characters processed. [BWTIncConstructFromPacked] 460 iterations done. 4600000000 characters processed. [BWTIncConstructFromPacked] 470 iterations done. 4700000000 characters processed. [BWTIncConstructFromPacked] 480 iterations done. 4800000000 characters processed. [BWTIncConstructFromPacked] 490 iterations done. 4900000000 characters processed. [BWTIncConstructFromPacked] 500 iterations done. 5000000000 characters processed. [BWTIncConstructFromPacked] 510 iterations done. 5100000000 characters processed. [BWTIncConstructFromPacked] 520 iterations done. 5200000000 characters processed. [BWTIncConstructFromPacked] 530 iterations done. 5300000000 characters processed. [BWTIncConstructFromPacked] 540 iterations done. 5400000000 characters processed. [BWTIncConstructFromPacked] 550 iterations done. 5500000000 characters processed. [BWTIncConstructFromPacked] 560 iterations done. 5600000000 characters processed. [BWTIncConstructFromPacked] 570 iterations done. 5700000000 characters processed. [BWTIncConstructFromPacked] 580 iterations done. 5798188880 characters processed. [BWTIncConstructFromPacked] 590 iterations done. 5886472096 characters processed. [BWTIncConstructFromPacked] 600 iterations done. 5964934432 characters processed. [BWTIncConstructFromPacked] 610 iterations done. 6034667936 characters processed. [BWTIncConstructFromPacked] 620 iterations done. 6096643264 characters processed. [BWTIncConstructFromPacked] 630 iterations done. 6151723072 characters processed. [BWTIncConstructFromPacked] 640 iterations done. 6200674128 characters processed. [BWTIncConstructFromPacked] 650 iterations done. 6244177920 characters processed. [BWTIncConstructFromPacked] 660 iterations done. 6282840176 characters processed. [BWTIncConstructFromPacked] 670 iterations done. 6317199264 characters processed. [BWTIncConstructFromPacked] 680 iterations done. 6347733664 characters processed. [BWTIncConstructFromPacked] 690 iterations done. 6374868704 characters processed. [BWTIncConstructFromPacked] 700 iterations done. 6398982368 characters processed. [BWTIncConstructFromPacked] 710 iterations done. 6418915856 characters processed. [bwt_gen] Finished constructing BWT in 710 iterations. [bwa_index] 3115.88 seconds elapse. [bwa_index] Update BWT... 24.15 sec [bwa_index] Pack forward-only FASTA... 21.30 sec [bwa_index] Construct SA from BWT and Occ... 1092.00 sec [main] Version: 0.7.12-r1039 [main] CMD: bwa index GCA_000001405.15_GRCh38_full_analysis_set.fna [main] Real time: 4647.870 sec; CPU: 4286.403 sec
hadoop@Mcnode3:~/cloud/adam/xubo/data/test20160422$ ll -h total 8.3G drwxrwxr-x 2 hadoop hadoop 4.0K 4月 22 16:42 ./ drwxrwxr-x 4 hadoop hadoop 4.0K 4月 22 15:22 ../ -rw------- 1 hadoop hadoop 3.1G 4月 22 15:24 GCA_000001405.15_GRCh38_full_analysis_set.fna -rw-rw-r-- 1 hadoop hadoop 20K 4月 22 16:21 GCA_000001405.15_GRCh38_full_analysis_set.fna.amb -rw-rw-r-- 1 hadoop hadoop 72K 4月 22 16:21 GCA_000001405.15_GRCh38_full_analysis_set.fna.ann -rw-rw-r-- 1 hadoop hadoop 3.0G 4月 22 16:19 GCA_000001405.15_GRCh38_full_analysis_set.fna.bwt -rw-rw-r-- 1 hadoop hadoop 766M 4月 22 16:21 GCA_000001405.15_GRCh38_full_analysis_set.fna.pac -rw-rw-r-- 1 hadoop hadoop 1.5G 4月 22 16:42 GCA_000001405.15_GRCh38_full_analysis_set.fna.sa