Skip to content

Commit

Permalink
Marks the IonInputFormat as not splittable, meaning that Ion files wi…
Browse files Browse the repository at this point in the history
…ll not be split into arbitrarily-sized chunks.

Without additional logic (which is possible to write, but is currently not written) to ensure each chunk contains
a self-contained symbol table context, splitting Ion files will cause errors on read.
  • Loading branch information
tgregg committed Jun 10, 2021
1 parent ac7ca23 commit 3bf3aef
Showing 1 changed file with 5 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,11 @@ public class IonInputFormat extends FileInputFormat {

private static final Log LOG = LogFactory.getLog(IonInputFormat.class);

@Override
protected boolean isSplitable(final FileSystem fs, final Path filename) {
return false;
}

@Override
public RecordReader getRecordReader(final InputSplit split, final JobConf job, final Reporter reporter)
throws IOException {
Expand Down

0 comments on commit 3bf3aef

Please sign in to comment.