Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to debug the compilation error "Cannot extract value from singleton.ops.impl.OpMacro"? #152

Open
tribbloid opened this issue Jul 29, 2020 · 24 comments

Comments

@tribbloid
Copy link

tribbloid commented Jul 29, 2020

Hi @fthomas @soronpo Thanks a lot for the great work, I'm one of the supporting developer of the deep learning library kotlingrad. When we try to integrate singleton-ops for the project on some tensor operations, we encounter some strange compiler behaviours that are hard to isolate or simplify, so we have to post a link to our repository in case others want to reproduce, sorry.

In one of our test cases: https://github.com/tribbloid/shapesafe/blob/59405baedc9048a9cac1fbf2bb7708c917266a6e/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala#L167

The compiler / macro misbehave on 2 apparently equivalent code blocks:

      val v0 = DoubleVector.random(6)
      val v1 = v0.pad(3)

      {
        val result = v1.reify

        val aa = result.arity

        {
          print_@(WideTyped(result.arity).viz)
          // this works
          result.arity.internal.dummyImp(3)
          result.crossValidate()
          result.arity.internal.requireEqual(12)
        }

        {
          print_@(WideTyped(aa).viz)
          // this doesn't, how did it happened?
          aa.internal.dummyImp(3)
          aa.internal.requireEqual(12)
        }
      }

These are the compilation errors:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:188: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))): OpMacro[+, S, Int, Int]
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:189: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.Require,singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.==,S,Int(12),Int(0)],String("Cannot prove requirement Require[...]"),singleton.ops.impl.NoSym]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.Require, List()), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$eq$eq, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(12)), FoldableConstantType(Constant(0)))), FoldableConstantType(Constant(Cannot prove requirement Require[...])), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.NoSym, List())))): OpMacro[Require, OpMacro[==, S, Int, Int], String, NoSym]
two errors found

> Task :core:compileTestScala FAILED

These are the same error if the implicit debugging plugin "splain" (https://github.com/tek/splain) is disabled:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:188: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:189: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.Require,singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.==,S,Int(12),Int(0)],String("Cannot prove requirement Require[...]"),singleton.ops.impl.NoSym]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.Require, List()), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$eq$eq, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(12)), FoldableConstantType(Constant(0)))), FoldableConstantType(Constant(Cannot prove requirement Require[...])), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.NoSym, List())))
two errors found

> Task :core:compileTestScala FAILED

My questions are:

  1. What's the cause of this error? It clearly won't be caused by a missing dependent type (variable aa has a static path, but result.arity doesn't) So how is the macro unable to figure out the type for aa?

  2. What should we do for any error that has such message? Should we debug or try to print the stacktrace?

@tribbloid
Copy link
Author

The full context of this error is displayed in the CI output:

https://github.com/tribbloid/shapesafe/runs/925426505?check_suite_focus=true#step:4:76

@soronpo
Copy link
Collaborator

soronpo commented Jul 30, 2020

From what I see in the error you have an operation S + 3, but S is unknown. So you are loosing refinement somewhere along the way.
Maybe it's possible to improve the error messages.

@tribbloid
Copy link
Author

That's the obvious part, the no-so-obvious part is why result.arity has the refinement correctly.

Also, the S should refer to result.arity.S theoretically, let me try to infer its full type hierarchy using reflection ...

@tribbloid
Copy link
Author

The result.arity.S can't generate a TypeTag automatically, so I print the type hierarchy of result.arity instead:

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]#Out#Out
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Invar[Int(6)]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Out_=[edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)]]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Arity
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proven
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proof
	- java.io.Serializable
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Operand
	- Object
	- Any

The type definition on the root level is Arity.FromLiteral[S] which means S resolves to Witness narrow type Int(6), I hope this is refined enough for you?

@soronpo
Copy link
Collaborator

soronpo commented Jul 30, 2020

Can you please do the following:

  1. clone the singleton-ops library.
  2. flip the VerboseTravel flag https://github.com/fthomas/singleton-ops/blob/master/src/main/scala/singleton/ops/impl/GeneralMacros.scala#L342
  3. publish it locally.
  4. configure your project to rely on the locally publised singleton-ops.
  5. recompile your example and paste the compilation log here (first have a successful compilation, and then uncomment the problematic code and get just incremental compilation log).

Afterwords, go back to the original configuration. The log is huge and will slow you down.

@tribbloid
Copy link
Author

Thanks a lot, here is the relevant part of the log (logging level set to --info):

@@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
-->
  @@OpCalc@@
  TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
  RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
---->
    @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
    TP: 
    RAW: TypeBounds(TypeRef(ThisType(scala), scala.Nothing, List()), TypeRef(ThisType(scala), scala.Any, List()))
------>
      @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
      TP: Any
      RAW: TypeRef(ThisType(scala), scala.Any, List())
-------->
        @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
        TP:  {
          final def ==(x$1: Any): Boolean
          final def !=(x$1: Any): Boolean
          def equals(x$1: Any): Boolean
          def hashCode(): Int
          def toString(): String
          def getClass(): Class[_]
          final def isInstanceOf[T0]: Boolean
          final def asInstanceOf[T0]: T0
          final def ## : Int
        }
        RAW: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
---------->
          Exhausted search at: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
<----------
<--------
<------
<----
  @@Unknown@@
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: Int(3)
  RAW: FoldableConstantType(Constant(3))
---->
<----
  Exhausted search at: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
<--
@@Unknown@@
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
!!!!!!aborted with: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0)))) at Some(trait OpMacro), Some(trait OpMacro)
@@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
-->
  @@OpCalc@@
  TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
  RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
---->
    @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
    TP: 
    RAW: TypeBounds(TypeRef(ThisType(scala), scala.Nothing, List()), TypeRef(ThisType(scala), scala.Any, List()))
------>
      @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
      TP: Any
      RAW: TypeRef(ThisType(scala), scala.Any, List())
-------->
        @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
        TP:  {
          final def ==(x$1: Any): Boolean
          final def !=(x$1: Any): Boolean
          def equals(x$1: Any): Boolean
          def hashCode(): Int
          def toString(): String
          def getClass(): Class[_]
          final def isInstanceOf[T0]: Boolean
          final def asInstanceOf[T0]: T0
          final def ## : Int
        }
        RAW: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
---------->
          Exhausted search at: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
<----------
<--------
<------
<----
  @@Unknown@@
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: Int(3)
  RAW: FoldableConstantType(Constant(3))
---->
<----
  Exhausted search at: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
<--
@@Unknown@@
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
!!!!!!aborted with: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0)))) at Some(trait OpMacro), Some(trait OpMacro)
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:185: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))): OpMacro[+, S, Int, Int]

Hope it could be useful

@tribbloid
Copy link
Author

@soronpo Thanks a lot for your attention, did you observe anything strange in the log?

@soronpo
Copy link
Collaborator

soronpo commented Aug 8, 2020

Thanks for the reminder. Forgot to respond. From the looks of it, indeed it's not singleton-ops territory. From the getgo the S is fed as an argument and there is nothing we can do about it. So the refinement is lost at the use-site. Maybe running the compiler with -Vtyper would yield some more information for you to understand where the refinement is lost.

@tribbloid
Copy link
Author

It looks like the Vtyper feature is highly experimental: I got the following error on a very ordinary generic case class:

caught scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving method copy: while typing new VizType

@soronpo
Copy link
Collaborator

soronpo commented Aug 10, 2020

It looks like the Vtyper feature is highly experimental: I got the following error on a very ordinary generic case class:

caught scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving method copy: while typing new VizType

Try with -Ytyper-debug instead

@tribbloid
Copy link
Author

-Ytyper-debug gave almost identical error. The full scope of the error looks like this:

(I guess this is more suitable to be posted on scalabug issue tracker, bu here you go)

|    |    |    |    |-- new VizType(tpe) EXPRmode (site: method copy in VizType) 
|    |    |    |    |    |-- new VizType APPSELmode-BYVALmode-EXPRmode-FUNmode-POLYmode (silent: method copy in VizType) 
|    |    |    |    |    |    |-- new VizType APPSELmode-EXPRmode-POLYmode-QUALmode (silent: method copy in VizType) 
|    |    |    |    |    |    |    |    caught scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving method copy: while typing new VizType
## Exception when compiling 7 sources to /home/peng/git/shapesafe/graph-commons/build/classes/scala/main
java.lang.AssertionError: assertion failed: (VizType,case class WideTyped[T] extends scala.Product with scala.Serializable {
  <caseaccessor> <paramaccessor> private[this] val value: T = _;
  def <init>(value: T): graph.commons.util.WideTyped[T] = {
    super.<init>();
    ()
  };
  type Wide = T;
  def viz(implicit ttag: TypeTag[T]): graph.commons.util.viz.VizType = VizType[T]
})
scala.reflect.internal.SymbolTable.throwAssertionError(SymbolTable.scala:170)
scala.tools.nsc.typechecker.TypersTracking$typingStack$.pop(TypersTracking.scala:98)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6026)
scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:6051)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$8(Typers.scala:3397)
scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3397)
scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5582)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5885)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5973)
scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.apply(Analyzer.scala:117)
scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:454)
scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.run(Analyzer.scala:105)
scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1515)
scala.tools.nsc.Global$Run.compileUnits(Global.scala:1499)
scala.tools.nsc.Global$Run.compileSources(Global.scala:1491)
scala.tools.nsc.Global$Run.compile(Global.scala:1626)
xsbt.CachedCompiler0.run(CompilerInterface.scala:153)
xsbt.CachedCompiler0.run(CompilerInterface.scala:125)
xsbt.CompilerInterface.run(CompilerInterface.scala:39)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:248)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:122)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:95)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:91)
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:186)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3(MixedAnalyzingCompiler.scala:82)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3$adapted(MixedAnalyzingCompiler.scala:77)
sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:215)
sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:77)
sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:146)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:343)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:343)
sbt.internal.inc.Incremental$.doCompile(Incremental.scala:120)
sbt.internal.inc.Incremental$.$anonfun$compile$4(Incremental.scala:100)
sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:180)
sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:98)
sbt.internal.inc.Incremental$.$anonfun$compile$3(Incremental.scala:102)
sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:155)
sbt.internal.inc.Incremental$.compile(Incremental.scala:92)
sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:75)
sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:348)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:301)
sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:168)
sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:248)
sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:74)
org.gradle.api.internal.tasks.scala.ZincScalaCompiler.execute(ZincScalaCompiler.java:147)
org.gradle.api.internal.tasks.scala.ZincScalaCompilerFacade.execute(ZincScalaCompilerFacade.java:47)
org.gradle.api.internal.tasks.scala.ZincScalaCompilerFacade.execute(ZincScalaCompilerFacade.java:31)
org.gradle.api.internal.tasks.compile.daemon.AbstractDaemonCompiler$CompilerWorkAction.execute(AbstractDaemonCompiler.java:135)
org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:63)
org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:49)
org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:43)
org.gradle.internal.classloader.ClassLoaderUtils.executeInClassloader(ClassLoaderUtils.java:97)
org.gradle.workers.internal.AbstractClassLoaderWorker.executeInClassLoader(AbstractClassLoaderWorker.java:43)
org.gradle.workers.internal.IsolatedClassloaderWorker.run(IsolatedClassloaderWorker.java:49)
org.gradle.workers.internal.IsolatedClassloaderWorker.run(IsolatedClassloaderWorker.java:30)
org.gradle.workers.internal.WorkerDaemonServer.run(WorkerDaemonServer.java:85)
org.gradle.workers.internal.WorkerDaemonServer.run(WorkerDaemonServer.java:55)
org.gradle.process.internal.worker.request.WorkerAction$1.call(WorkerAction.java:138)
org.gradle.process.internal.worker.child.WorkerLogEventListener.withWorkerLoggingProtocol(WorkerLogEventListener.java:41)
org.gradle.process.internal.worker.request.WorkerAction.run(WorkerAction.java:135)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
java.lang.Thread.run(Thread.java:748)
           

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':graph-commons:compileScala'.
> assertion failed: (VizType,case class WideTyped[T] extends scala.Product with scala.Serializable {
    <caseaccessor> <paramaccessor> private[this] val value: T = _;
    def <init>(value: T): graph.commons.util.WideTyped[T] = {
      super.<init>();
      ()
    };
    type Wide = T;
    def viz(implicit ttag: TypeTag[T]): graph.commons.util.viz.VizType = VizType[T]
  })

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 14s

@soronpo
Copy link
Collaborator

soronpo commented Aug 10, 2020

Maybe it's a compiler flag. Maybe some instability as some improper macro use. TypeTag stuff maybe make things problematic.

@tribbloid
Copy link
Author

tribbloid commented Aug 12, 2020

agreed, after spending some time playing with the compiler it appears that all implicit inference from symbols like S has the same problem, if I replace the line that causes the error with:

universe.typeTag[aa.SS]

I got a similar error:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:197: implicit error;
!I ttag (No TypeTag available for Invar.this.SS): TypeTag[S]

This in the end can be traced back to a alleged compiler bug:

https://stackoverflow.com/questions/59708880/in-scala-why-it-is-impossible-to-infer-typetag-from-type-alias-or-dependent-typ

Of which the cause is still inconclusive. Let me try some other tricks before figuring out a way to bypass it

@tribbloid
Copy link
Author

No it was not relevant, original ticket (scala/scala#5836) merely shows a feature being removed due to conflict with other behaviour of the compiler.

So I end up debugging it in runtime using reflection:

          import ScalaReflection.universe

          def sniff[T](v: T)(implicit ttag: universe.TypeTag[T]) = ttag

          val aaT = sniff(aa).tpe
          print(universe.showRaw(aaT.typeSymbol.typeSignature))

Which gives the proper type argument you have observed:

PolyType(List(TypeName("S")), ClassInfoType(List(TypeRef(ThisType(scala), TypeName("AnyRef"), List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const, List(TypeRef(NoPrefix, TypeName("S"), List())))), Scope(TermName("singleton"), TermName("singleton "), termNames.CONSTRUCTOR, TermName("number")), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral))

But it is before the resolution! Using the baseType of the type directly (instead of the type signature) reveals the true nature of the type argument:

          val sT = aaT.baseType(aaT.baseClasses.head).typeArgs.head

          print(universe.showRaw(sT))

gives the refined type:

FoldableConstantType(Constant(6))

Of course, I was conducting the experiment in runtime universe, and the compile-time may follow an entirely different path. Regardless, do you think the above baseType trick can be used in your code? Do you have a test class responsible for OpMacro summoning on edge cases like this? I'll see if I can compile it into shorter and more reproducible code test cases. Hopefully we can get a patch ready ASAP.

@soronpo
Copy link
Collaborator

soronpo commented Aug 12, 2020

Can you post a print of the aaT.typeSymbol.typeSignature without the showRaw?

@tribbloid
Copy link
Author

yes sir, println(aaT.typeSymbol.typeSignature) yields:

[S <: Int]AnyRef
        with edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[S] {
  val singleton: S
  private[this] val singleton: S
  def <init>(singleton: S): edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[S]
  override def number: Int
}

BTW println(sT) yields:

Int(6)

@soronpo
Copy link
Collaborator

soronpo commented Aug 13, 2020

If you will be able to minimize this, it would help a lot. Can you please provide the following:

println(aaT)
println(universe.showRaw(aaT))
println(aaT.dealias)
println(universe.showRaw(aaT.dealias))
println(aaT.typeSymbol)
println(universe.showRaw(aaT.typeSymbol))
println(aaT.baseType)
println(universe.showRaw(aaT.baseType))

@tribbloid
Copy link
Author

At your service:

          println(aaT)

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]#Out#Out

          println(universe.showRaw(aaT))

TypeRef(TypeRef(TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral, List(FoldableConstantType(Constant(6)))), TypeName("Out"), List()), TypeName("Out"), List())

          println(aaT.dealias)

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]

          println(universe.showRaw(aaT.dealias))

TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral, List(FoldableConstantType(Constant(6))))

          println(aaT.typeSymbol)

class FromLiteral

          println(universe.showRaw(aaT.typeSymbol))

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral

          println(
            aaT.baseClasses.map { clazz =>
              val baseType = aaT.baseType(clazz)
              baseType
            }
          )

List(edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)], edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)], edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Invar[Int(6)], edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Out_=[edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)]], edu.umontreal.kotlingrad.shapesafe.m.arity.Arity, edu.umontreal.kotlingrad.shapesafe.m.arity.Proven, edu.umontreal.kotlingrad.shapesafe.m.arity.Proof, java.io.Serializable, edu.umontreal.kotlingrad.shapesafe.m.arity.Operand, Object, Any)

          println(
            aaT.baseClasses.map { clazz =>
              val baseType = aaT.baseType(clazz)
              universe.showRaw(baseType)
            }
          )

List(TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral, List(FoldableConstantType(Constant(6)))), TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const, List(FoldableConstantType(Constant(6)))), TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Invar, List(FoldableConstantType(Constant(6)))), TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Out_$eq, List(TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const, List(FoldableConstantType(Constant(6)))))), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity, List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proven, List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof, List()), TypeRef(ThisType(java.io), java.io.Serializable, List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Operand, List()), TypeRef(ThisType(java.lang), java.lang.Object, List()), TypeRef(ThisType(scala), scala.Any, List()))

The following 2 phrases doesn't compile:

println(aaT.baseType)
println(universe.showRaw(aaT.baseType))

so I twisted them a bit.

@tribbloid
Copy link
Author

@soronpo
Copy link
Collaborator

soronpo commented Aug 13, 2020

The 6 type value isn't making its way into singleton-ops, so it looks like the typetag is inspecting the wrong thing.
For example. If FromLiteral was defined like so:

trait FromtLiteral[T] {
  type Out = T
}
val fl : FromLiteral[6] = ???

Then you implicitly[fl.Out + 3] would have worked for sure.

@tribbloid
Copy link
Author

@soronpo sorry for the slow reply, I just tried your example and it reproduces the error:

object ErrorCase0 {

  case class Outer[S]() {

    type SS = S
  }

  val f1 = Outer[Witness.`6`]()

  implicitly[f1.SS + Witness.`3`]
}

if compiled with either scala 2.12.11 or 2.13.3, the compile reports:

[Error] /home/peng/git-spike/scalaspike/common/src/test/scala/com/tribbloids/spike/singleton_ops_spike/ErrorCase0.scala:15: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,Any,Any,singleton.ops.NP]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(ThisType(scala), scala.Any, List()), TypeRef(ThisType(scala), scala.Any, List()), TypeRef(SingleType(SingleType(SingleType(ThisType(<root>), singleton), singleton.ops), singleton.ops.package), TypeName("NP"), List())))

@tribbloid
Copy link
Author

If you like to play with it, voila:

https://scastie.scala-lang.org/tribbloid/3Ijg2dSLRcmiiNrXnSf4hg

@soronpo
Copy link
Collaborator

soronpo commented Aug 29, 2020

You need W.`XXX`.T
https://scastie.scala-lang.org/pL4vfrNISziWcKHUBlX8fA

@tribbloid
Copy link
Author

@soronpo I'm so sorry, after 2 weeks I no longer remember how shapeless works 🤦

I think I've finally isolate the problem, all caused by this.type as an abstract type:

import shapeless.Witness
import singleton.ops.+

import scala.language.implicitConversions

object ErrorCase0 {

  trait Proof extends Serializable {

    type Out <: Arity
    def out: Out
  }

  trait Arity extends Proof {

    override type Out = this.type
    override def out: Out = this
  }

  object Arity {

    trait Const[S] extends Arity with Proof {

      type SS = S
    }

    case class FromLiteral[S <: Int]() extends Const[S] {}
  }

  val v1 = Arity.FromLiteral[Witness.`6`.T]()
//  val f1 = Arity(6)
  implicitly[v1.SS + Witness.`3`.T]

  val v2 = v1.out
  implicitly[v2.SS + Witness.`3`.T] // this fails
}

https://scastie.scala-lang.org/tribbloid/yRXvkpWWRiWAyGHzpI9g1Q/1

Scalac was known for fumbling on this.type inference. This issue may be one instance of such fumbling and may won't be fixed without some major compiler overhaul. Regardless, the above case may worth further investigation to be 100% sure of the cause

For the moment I'll just replace the abstract type with an F-bounded type parameter to circumvent it. Thank you so much for your instructions!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants