Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java.lang.NoClassDefFoundError: scala/Product$class #79

Open
jsubm5 opened this issue Mar 6, 2023 · 0 comments
Open

java.lang.NoClassDefFoundError: scala/Product$class #79

jsubm5 opened this issue Mar 6, 2023 · 0 comments

Comments

@jsubm5
Copy link

jsubm5 commented Mar 6, 2023

I'm trying to extract data form Salesforce via Spark in databricks
When doing a spark.read, getting "java.lang.NoClassDefFoundError: scala/Product$class"

soql = "select id from opportunity where 1=2"
sfDF = (spark.
read.
format("com.springml.spark.salesforce").
option("username", SFDCusername).
option("password", SFDCPwd+SecToken).
option("soql", soql).
option("version", "37.0").
load())

Databricks version : 11.3 LTS (includes Apache Spark 3.3.0, Scala 2.12)
Here are the dependencies installed to databricks clusters
com.springml:spark-salesforce_2.11:1.1.3
com.springml:salesforce-wave-api:1.0.10

Here is the detailed error
java.lang.NoClassDefFoundError: scala/Product$class

Py4JJavaError Traceback (most recent call last)
in <cell line: 2>()
1 soql = "select id from opportunity where 1=2"
----> 2 sfDF = (spark.
3 read.
4 format("com.springml.spark.salesforce").
5 option("username", SFDCusername).

/databricks/spark/python/pyspark/instrumentation_utils.py in wrapper(*args, **kwargs)
46 start = time.perf_counter()
47 try:
---> 48 res = func(*args, **kwargs)
49 logger.log_success(
50 module_name, class_name, function_name, time.perf_counter() - start, signature

/databricks/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options)
182 return self._df(self._jreader.load(self._spark._sc._jvm.PythonUtils.toSeq(path)))
183 else:
--> 184 return self._df(self._jreader.load())
185
186 def json(

/databricks/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py in call(self, *args)
1319
1320 answer = self.gateway_client.send_command(command)
-> 1321 return_value = get_return_value(
1322 answer, self.gateway_client, self.target_id, self.name)
1323

/databricks/spark/python/pyspark/sql/utils.py in deco(*a, **kw)
194 def deco(*a: Any, **kw: Any) -> Any:
195 try:
--> 196 return f(*a, **kw)
197 except Py4JJavaError as e:
198 converted = convert_exception(e.java_exception)

/databricks/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name)
324 value = OUTPUT_CONVERTER[type](answer[2:], gateway_client)
325 if answer[1] == REFERENCE_TYPE:
--> 326 raise Py4JJavaError(
327 "An error occurred while calling {0}{1}{2}.\n".
328 format(target_id, ".", name), value)

Py4JJavaError: An error occurred while calling o655.load.
: java.lang.NoClassDefFoundError: scala/Product$class
at com.springml.spark.salesforce.DatasetRelation.(DatasetRelation.scala:35)
at com.springml.spark.salesforce.DefaultSource.createRelation(DefaultSource.scala:99)
at com.springml.spark.salesforce.DefaultSource.createRelation(DefaultSource.scala:50)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:387)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:375)
at org.apache.spark.sql.DataFrameReader.$anonfun$load$2(DataFrameReader.scala:331)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:331)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)
at py4j.Gateway.invoke(Gateway.java:306)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:195)
at py4j.ClientServerConnection.run(ClientServerConnection.java:115)
at java.lang.Thread.run(Thread.java:750)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant