Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stop at the same region of genome #86

Open
DustinChen1986 opened this issue Apr 23, 2021 · 1 comment
Open

Stop at the same region of genome #86

DustinChen1986 opened this issue Apr 23, 2021 · 1 comment

Comments

@DustinChen1986
Copy link

DustinChen1986 commented Apr 23, 2021 •
edited
Loading

Hi,
I got bam with pbmm2 v1.3.0.
pbmm2 align ref.mmi mymovies.fofn pbmm2.bam --sort
Then I used kineticsTools:
ipdSummary pbmm2.bam --reference genome.fasta --identify m6A,m4C --methylFraction --gff kinetics.gff --csv kinetics.csv --bigwig kinetics.bw --log-level INFO --numWorkers 12 --referenceStride 100000
ipdSummary started with:

[INFO] 2021-04-23 06:34:05,878Z [kineticsTools.ipdSummary _pacbio_main_runner 160] Using pbcommand v2.2.2
[INFO] 2021-04-23 06:34:05,878Z [kineticsTools.ipdSummary _pacbio_main_runner 161] completed setting up logger with <function setup_log at 0x7fd03a301ee0>
[INFO] 2021-04-23 06:34:05,878Z [kineticsTools.ipdSummary _pacbio_main_runner 164] log opts {'level': 20, 'file_name': None}
[INFO] 2021-04-23 06:34:05,881Z [root loadSharedAlignmentSet 475] Reading AlignmentSet: pbmm2.bam
[INFO] 2021-04-23 06:34:05,881Z [root loadSharedAlignmentSet 476] reference: /mnt/hgfs/G/1.Pacbio/genome.fasta
[WARNING] 2021-04-23 06:34:09,505Z [pbcore.io.dataset.DataSetIO _openFiles 2162] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[WARNING] 2021-04-23 06:34:09,860Z [pbcore.io.dataset.DataSetIO _openFiles 2162] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:09,987Z [kineticsTools.loader getIpdModelFilename 42] Using chemistry-matched kinetics model: '/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/resources/SP2-C2.npz.gz'
[INFO] 2021-04-23 06:34:09,987Z [root loadReferenceAndModel 463] Loading reference contigs '/mnt/hgfs/G/1.Pacbio/genome.fasta'
[INFO] 2021-04-23 06:34:24,769Z [root _launchSlaveProcesses 410] Available CPUs: 12
[INFO] 2021-04-23 06:34:24,770Z [root _launchSlaveProcesses 411] Requested worker processes: 12
[INFO] 2021-04-23 06:34:24,819Z [root _run 90] Worker KineticWorkerProcess-1 (PID=97866) started running
[INFO] 2021-04-23 06:34:24,826Z [root _run 90] Worker KineticWorkerProcess-2 (PID=97867) started running
[INFO] 2021-04-23 06:34:24,846Z [root _run 90] Worker KineticWorkerProcess-3 (PID=97868) started running
[INFO] 2021-04-23 06:34:24,850Z [root _run 90] Worker KineticWorkerProcess-5 (PID=97870) started running
[INFO] 2021-04-23 06:34:24,854Z [root _run 90] Worker KineticWorkerProcess-6 (PID=97871) started running
[INFO] 2021-04-23 06:34:24,858Z [root _run 90] Worker KineticWorkerProcess-4 (PID=97869) started running
[INFO] 2021-04-23 06:34:24,865Z [root _launchSlaveProcesses 437] Launched worker processes.
[INFO] 2021-04-23 06:34:24,866Z [root _run 90] Worker KineticWorkerProcess-7 (PID=97872) started running
[INFO] 2021-04-23 06:34:24,871Z [root _launchSlaveProcesses 443] Launched result collector process.
[INFO] 2021-04-23 06:34:24,873Z [root _run 90] Worker KineticWorkerProcess-8 (PID=97873) started running
[INFO] 2021-04-23 06:34:24,873Z [root _run 90] Worker KineticWorkerProcess-9 (PID=97874) started running
[INFO] 2021-04-23 06:34:24,880Z [kineticsTools.ResultWriter _run 40] Process KineticsWriter-13 (PID=97878) started running
[INFO] 2021-04-23 06:34:24,882Z [root _mainLoop 540] Generating kinetics summary for [pbmm2.bam]
[INFO] 2021-04-23 06:34:24,883Z [root _mainLoop 555] Processing window/contig: ReferenceWindow(refId=0, refName='Chr1', start=0, end=44929686)
[INFO] 2021-04-23 06:34:24,886Z [root _run 90] Worker KineticWorkerProcess-12 (PID=97877) started running
[INFO] 2021-04-23 06:34:24,892Z [root _run 90] Worker KineticWorkerProcess-10 (PID=97875) started running
[INFO] 2021-04-23 06:34:24,899Z [root _run 90] Worker KineticWorkerProcess-11 (PID=97876) started running
[INFO] 2021-04-23 06:34:26,389Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,410Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,413Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,426Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,443Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,458Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,467Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,468Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,511Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,551Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,599Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:26,619Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality
[INFO] 2021-04-23 06:34:27,174Z [root _run 125] Got chunk: (0, ReferenceWindow(refId=0, refName='Chr1', start=0, end=100000)) -- Process:
[INFO] 2021-04-23 06:34:27,418Z [root _run 125] Got chunk: (1, ReferenceWindow(refId=0, refName='Chr1', start=100000, end=200000)) -- Process:
[INFO] 2021-04-23 06:34:27,426Z [root _run 125] Got chunk: (2, ReferenceWindow(refId=0, refName='Chr1', start=200000, end=300000)) -- Process:
[INFO] 2021-04-23 06:34:27,450Z [root _run 125] Got chunk: (3, ReferenceWindow(refId=0, refName='Chr1', start=300000, end=400000)) -- Process:
[INFO] 2021-04-23 06:34:27,476Z [root _run 125] Got chunk: (6, ReferenceWindow(refId=0, refName='Chr1', start=600000, end=700000)) -- Process:
[INFO] 2021-04-23 06:34:27,476Z [root _run 125] Got chunk: (5, ReferenceWindow(refId=0, refName='Chr1', start=500000, end=600000)) -- Process:
[INFO] 2021-04-23 06:34:27,479Z [root _run 125] Got chunk: (7, ReferenceWindow(refId=0, refName='Chr1', start=700000, end=800000)) -- Process:
[INFO] 2021-04-23 06:34:27,483Z [root _run 125] Got chunk: (4, ReferenceWindow(refId=0, refName='Chr1', start=400000, end=500000)) -- Process:
[INFO] 2021-04-23 06:34:28,286Z [root _run 125] Got chunk: (8, ReferenceWindow(refId=0, refName='Chr1', start=800000, end=900000)) -- Process:
[INFO] 2021-04-23 06:34:28,659Z [root _run 125] Got chunk: (9, ReferenceWindow(refId=0, refName='Chr1', start=900000, end=1000000)) -- Process:
[INFO] 2021-04-23 06:34:28,862Z [root _summarizeReferenceRegion 226] Making summary: -15 to 100015
[INFO] 2021-04-23 06:34:29,190Z [root _summarizeReferenceRegion 226] Making summary: 799985 to 900015
[INFO] 2021-04-23 06:34:29,642Z [root _summarizeReferenceRegion 226] Making summary: 299985 to 400015
[INFO] 2021-04-23 06:34:29,717Z [root _summarizeReferenceRegion 226] Making summary: 99985 to 200015
[INFO] 2021-04-23 06:34:29,781Z [root _summarizeReferenceRegion 226] Making summary: 599985 to 700015
[INFO] 2021-04-23 06:34:29,835Z [root _run 125] Got chunk: (10, ReferenceWindow(refId=0, refName='Chr1', start=1000000, end=1100000)) -- Process:
[INFO] 2021-04-23 06:34:29,942Z [root _summarizeReferenceRegion 226] Making summary: 499985 to 600015
[INFO] 2021-04-23 06:34:29,978Z [root _run 125] Got chunk: (11, ReferenceWindow(refId=0, refName='Chr1', start=1100000, end=1200000)) -- Process:
[INFO] 2021-04-23 06:34:30,005Z [root _summarizeReferenceRegion 226] Making summary: 699985 to 800015
[INFO] 2021-04-23 06:34:30,088Z [root _summarizeReferenceRegion 226] Making summary: 399985 to 500015
[INFO] 2021-04-23 06:34:30,353Z [root _summarizeReferenceRegion 226] Making summary: 199985 to 300015
[INFO] 2021-04-23 06:34:31,961Z [root _summarizeReferenceRegion 226] Making summary: 899985 to 1000015
[INFO] 2021-04-23 06:34:32,743Z [root _summarizeReferenceRegion 226] Making summary: 999985 to 1100015
[INFO] 2021-04-23 06:34:32,954Z [root _summarizeReferenceRegion 226] Making summary: 1099985 to 1200015
[INFO] 2021-04-23 06:34:43,680Z [root _fetchChunks 426] Retrieved 584 hits
[INFO] 2021-04-23 06:34:47,165Z [root _fetchChunks 426] Retrieved 588 hits
[INFO] 2021-04-23 06:34:47,782Z [root _fetchChunks 426] Retrieved 553 hits
[INFO] 2021-04-23 06:34:49,053Z [root _fetchChunks 426] Retrieved 666 hits
[INFO] 2021-04-23 06:34:50,573Z [root _fetchChunks 426] Retrieved 693 hits
[INFO] 2021-04-23 06:34:53,061Z [root _fetchChunks 426] Retrieved 619 hits
[INFO] 2021-04-23 06:34:53,171Z [root _fetchChunks 426] Retrieved 709 hits
[INFO] 2021-04-23 06:34:53,553Z [root _fetchChunks 426] Retrieved 859 hits
[INFO] 2021-04-23 06:34:53,619Z [root _fetchChunks 426] Retrieved 800 hits
[INFO] 2021-04-23 06:34:54,199Z [root _fetchChunks 426] Retrieved 1126 hits
[INFO] 2021-04-23 06:34:56,197Z [root _fetchChunks 426] Retrieved 940 hits
[INFO] 2021-04-23 06:35:02,325Z [root _fetchChunks 426] Retrieved 1154 hits
[INFO] 2021-04-23 06:47:30,649Z [root _run 125] Got chunk: (12, ReferenceWindow(refId=0, refName='Chr1', start=1200000, end=1300000)) -- Process:
[INFO] 2021-04-23 06:47:33,234Z [root _run 125] Got chunk: (13, ReferenceWindow(refId=0, refName='Chr1', start=1300000, end=1400000)) -- Process:
[INFO] 2021-04-23 06:47:33,987Z [root _summarizeReferenceRegion 226] Making summary: 1199985 to 1300015
[INFO] 2021-04-23 06:47:37,258Z [root _summarizeReferenceRegion 226] Making summary: 1299985 to 1400015
[INFO] 2021-04-23 06:47:37,329Z [root _fetchChunks 426] Retrieved 1139 hits
[INFO] 2021-04-23 06:47:39,197Z [root _fetchChunks 426] Retrieved 631 hits
[INFO] 2021-04-23 06:47:42,357Z [root _run 125] Got chunk: (14, ReferenceWindow(refId=0, refName='Chr1', start=1400000, end=1500000)) -- Process:
[INFO] 2021-04-23 06:47:44,612Z [root _run 125] Got chunk: (15, ReferenceWindow(refId=0, refName='Chr1', start=1500000, end=1600000)) -- Process:
[INFO] 2021-04-23 06:47:46,085Z [root _run 125] Got chunk: (16, ReferenceWindow(refId=0, refName='Chr1', start=1600000, end=1700000)) -- Process:
[INFO] 2021-04-23 06:47:46,402Z [root _summarizeReferenceRegion 226] Making summary: 1399985 to 1500015
[INFO] 2021-04-23 06:47:48,111Z [root _fetchChunks 426] Retrieved 462 hits
[INFO] 2021-04-23 06:47:48,588Z [root _summarizeReferenceRegion 226] Making summary: 1499985 to 1600015
[INFO] 2021-04-23 06:47:49,867Z [root _summarizeReferenceRegion 226] Making summary: 1599985 to 1700015
[INFO] 2021-04-23 06:47:51,621Z [root _fetchChunks 426] Retrieved 1129 hits
[INFO] 2021-04-23 06:47:51,943Z [root _run 125] Got chunk: (17, ReferenceWindow(refId=0, refName='Chr1', start=1700000, end=1800000)) -- Process:
[INFO] 2021-04-23 06:47:54,628Z [root _fetchChunks 426] Retrieved 3289 hits
Process KineticWorkerProcess-12:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/WorkerProcess.py", line 151, in run
self._run()
File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/WorkerProcess.py", line 127, in _run
result = self.onChunk(
File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/KineticWorker.py", line 121, in onChunk
perSiteResults = self._summarizeReferenceRegion(
File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/KineticWorker.py", line 230, in _summarizeReferenceRegion
(caseChunks, capValue) = self._fetchChunks(
File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/KineticWorker.py", line 432, in _fetchChunks
hits = np.random.choice(
File "mtrand.pyx", line 895, in numpy.random.mtrand.RandomState.choice
File "/usr/local/lib/python3.8/dist-packages/pbcore-2.3.1-py3.8.egg/pbcore/io/align/BamAlignment.py", line 68, in f
raise UnavailableFeature(
pbcore.io.align._BamSupport.UnavailableFeature: this feature requires a PacBio BAM index
Child process exited with exitcode=1. Aborting.
[ERROR] 2021-04-23 06:47:55,550Z [root monitorChildProcesses 592] Child process exited with exitcode=1. Aborting.

Would you help me to fix it?
Thank you for your replication.
@DustinChen1986 DustinChen1986 changed the title Stop at the same region of genome with OSError Stop at the same region of genome Apr 23, 2021
@rhallPB
Copy link
Contributor

rhallPB commented Apr 23, 2021

The output suggests an issue with the aligned bam file, the first thing to check would be to generate the pbi index file for the bam https://pbbam.readthedocs.io/en/latest/tools/pbindex.html to see if this solves the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rhallPB @DustinChen1986