stack cradle does not discover .cabal files if they are deeper than 1 subdirectory

[jneira]

• from With Ormolu as the formatting provider, need BangPatterns (el al.) without putting it in every file haskell/haskell-language-server#382 (comment)
• project structure and tests done by @comlyue copied from the issue

cymp:edh-universe cyue$ cat stack.yaml

# Please change the resolver and other Stack settings per your needs
resolver: lts-16.11

ghc-options:
  elr: -O0
  edh: -O0

## FOLLOWING CONTENTS GONNA BE OVERWRITTEN BY EPM ##

local-bin-path: /fw/m3cyue/edh-universe/bin
packages:
 - /fw/m3cyue/edh-universe/e-wrks/edh/lossless-decimal
 - /fw/m3cyue/edh-universe/e-wrks/edh/elr
 - /fw/m3cyue/edh-universe/e-wrks/edh/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/nedh/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/sedh/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/hasdim/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/haskit/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/els/host.hs
cymp:edh-universe cyue$

cymp:edh-universe cyue$ find . -name \*.cabal
./e-wrks/edh/lossless-decimal/lossless-decimal.cabal
./e-wrks/edh/elr/elr.cabal
./e-wrks/edh/host.hs/edh.cabal
./e-wrks/nedh/host.hs/nedh.cabal
./e-wrks/sedh/host.hs/sedh.cabal
./e-wrks/hasdim/host.hs/hasdim.cabal
./e-wrks/haskit/host.hs/haskit.cabal
./e-wrks/els/host.hs/els.cabal
cymp:edh-universe cyue$

cymp:edh-universe cyue$ gen-hie
gen-hie: No .cabal files found under/fw/m3cyue/edh-universe
 You may need to run stack build.
CallStack (from HasCallStack):
  error, called at app/Main.hs:32:21 in main:Main
cymp:edh-universe cyue$ 

cymp:edh-universe cyue$ cd e-wrks/els/host.hs
cymp:host.hs cyue$ gen-hie
gen-hie: user error (InvalidYaml (Just (YamlException "Yaml file not found: /fw/m3cyue/edh-universe/e-wrks/els/host.hs/stack.yaml")))
cymp:host.hs cyue$ 

[Avi-D-coder]

@complyue Try deleting fw/m3cyue from your packages paths.
If absolute paths are not the issue, please produce a minimal repo, that I can debug with.

[complyue]

Turns out it's the presence of dist-newstyle caused the err out:

$ echo 'packages:' > stack.yaml 
$ gen-hie
cradle:
  stack:
$ mkdir dist-newstyle
$ gen-hie
gen-hie: No .cabal files found under/fw/ttt
 You may need to run stack build.
CallStack (from HasCallStack):
  error, called at app/Main.hs:22:5 in main:Main
$ 

[Avi-D-coder]

So is Every thing working now?
I'll add a message to that case explaining which cradle was used and why.

[complyue]

I still got empty list in my case even with .cabal files listed in my stack.yaml. I'm not hurry in pursuing, but if there's simple diagnostic actions, like verbose debug trace etc. I can tinker with it.

[Avi-D-coder]

I don't have time to add logging right now.
I added which tool is being used to the error.
Try with the master GitHub version and see if it's still using cabal.

Also just run gen-hie --cabal and post the output.

[complyue]

with gen-hie built from master:

cymp:edh-universe cyue$ cat stack.yaml

# Please change the resolver and other Stack settings per your needs
resolver: lts-16.11

ghc-options:
  elr: -O0
  edh: -O0

## FOLLOWING CONTENTS GONNA BE OVERWRITTEN BY EPM ##

local-bin-path: /fw/m3cyue/edh-universe/bin
packages:
 - /fw/m3cyue/edh-universe/e-wrks/edh/lossless-decimal
 - /fw/m3cyue/edh-universe/e-wrks/edh/elr
 - /fw/m3cyue/edh-universe/e-wrks/edh/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/nedh/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/sedh/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/hasdim/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/haskit/host.hs
 - /fw/m3cyue/edh-universe/e-wrks/els/host.hs
cymp:edh-universe cyue$ 
cymp:edh-universe cyue$ find . -name \*.cabal
./e-wrks/edh/lossless-decimal/lossless-decimal.cabal
./e-wrks/edh/elr/elr.cabal
./e-wrks/edh/host.hs/edh.cabal
./e-wrks/nedh/host.hs/nedh.cabal
./e-wrks/sedh/host.hs/sedh.cabal
./e-wrks/hasdim/host.hs/hasdim.cabal
./e-wrks/haskit/host.hs/haskit.cabal
./e-wrks/els/host.hs/els.cabal
cymp:edh-universe cyue$ 
cymp:edh-universe cyue$ gen-hie
cradle:
  stack:
cymp:edh-universe cyue$ 
cymp:edh-universe cyue$ gen-hie --cabal
gen-hie: Usedcabal
 No .cabal files found under/fw/m3cyue/edh-universe
 You may need to run stack build.
CallStack (from HasCallStack):
  error, called at app/Main.hs:22:5 in main:Main
cymp:edh-universe cyue$ 

[complyue]

Oh, turns out it matters for the package path to be relative, now:

cymp:edh-universe cyue$ cat stack.yaml

# Please change the resolver and other Stack settings per your needs
resolver: lts-16.11

ghc-options:
  elr: -O0
  edh: -O0

## FOLLOWING CONTENTS GONNA BE OVERWRITTEN BY EPM ##

local-bin-path: /fw/m3cyue/edh-universe/bin
packages:
 - ./e-wrks/edh/lossless-decimal
 - ./e-wrks/edh/elr
 - ./e-wrks/edh/host.hs
 - ./e-wrks/nedh/host.hs
 - ./e-wrks/sedh/host.hs
 - ./e-wrks/hasdim/host.hs
 - ./e-wrks/haskit/host.hs
 - ./e-wrks/els/host.hs
cymp:edh-universe cyue$ 
cymp:edh-universe cyue$ gen-hie
cradle:
  stack:
    - path: "./e-wrks/edh/elr/src"
      component: "elr:lib"

    - path: "./e-wrks/edh/host.hs/src"
      component: "edh:lib"

    - path: "./e-wrks/edh/host.hs/repl/Main.hs"
      component: "edh:exe:edh"

    - path: "./e-wrks/edh/host.hs/repl/Repl.hs"
      component: "edh:exe:edh"

    - path: "./e-wrks/edh/host.hs/runfile/Main.hs"
      component: "edh:exe:runedh"

    - path: "./e-wrks/edh/host.hs/runmodu/Main.hs"
      component: "edh:exe:edhm"

    - path: "./e-wrks/edh/lossless-decimal/src"
      component: "lossless-decimal:lib"

    - path: "./e-wrks/els/host.hs/src"
      component: "els:lib"

    - path: "./e-wrks/els/host.hs/server/Main.hs"
      component: "els:exe:els"

    - path: "./e-wrks/hasdim/host.hs/repl/Main.hs"
      component: "hasdim:exe:hasdim"

    - path: "./e-wrks/hasdim/host.hs/repl/Repl.hs"
      component: "hasdim:exe:hasdim"

    - path: "./e-wrks/hasdim/host.hs/runfile/Main.hs"
      component: "hasdim:exe:rundim"

    - path: "./e-wrks/hasdim/host.hs/runmodu/Main.hs"
      component: "hasdim:exe:rundimm"

    - path: "./e-wrks/hasdim/host.hs/src"
      component: "hasdim:lib"

    - path: "./e-wrks/haskit/host.hs/hski/Main.hs"
      component: "haskit:exe:hski"

    - path: "./e-wrks/haskit/host.hs/hski/Repl.hs"
      component: "haskit:exe:hski"

    - path: "./e-wrks/haskit/host.hs/src"
      component: "haskit:lib"

    - path: "./e-wrks/nedh/host.hs/src"
      component: "nedh:lib"

    - path: "./e-wrks/nedh/host.hs/runmodu/Main.hs"
      component: "nedh:exe:nedh"

    - path: "./e-wrks/nedh/host.hs/runmodu/Repl.hs"
      component: "nedh:exe:nedh"

    - path: "./e-wrks/nedh/host.hs/runfile/Main.hs"
      component: "nedh:exe:runnedh"

    - path: "./e-wrks/sedh/host.hs/gwd/Main.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/gwd/default-language:.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/gwd/Haskell2010.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/forage/Main.hs"
      component: "sedh:exe:forage"

    - path: "./e-wrks/sedh/host.hs/forage/default-language:.hs"
      component: "sedh:exe:forage"

    - path: "./e-wrks/sedh/host.hs/forage/Haskell2010.hs"
      component: "sedh:exe:forage"

    - path: "./e-wrks/sedh/host.hs/src"
      component: "sedh:lib"
cymp:edh-universe cyue$ 

I'll manage to have epm to generate relative package path later to workaround it.

But then I think I discovered another bug, that when other-modules: is empty, the next entry (i.e. default-language: Haskell2010) is mistaken for its content. E.g. seeing https://github.com/complyue/sedh/blob/b290f7532d1ad230f38684919bf6d7de851e6fa3/host.hs/sedh.cabal#L27-L28

executable gwd
  main-is:            Main.hs
  hs-source-dirs:     gwd
  other-modules:
  default-language:   Haskell2010
  default-extensions:
    NoImplicitPrelude
...

generating:

    - path: "./e-wrks/sedh/host.hs/gwd/Main.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/gwd/default-language:.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/gwd/Haskell2010.hs"
      component: "sedh:exe:gwd"

[complyue]

I removed empty other-modules: to workaround it, so now gen-hie can produce hie.yaml as good as a manual crafted one.

But hls still fails with Exception: BadDependency "GhcSession" on opening "./e-wrks/sedh/host.hs/gwd/Main.hs", though "./e-wrks/els/host.hs/server/Main.hs" works well, these 2 module and their respective .cabal file are very similar in structure, it feels weird one works but not the other. Anyway majority of the library source files work rather smooth, so I would postpone the cracking on this issue later. At least after I make the initial release of these packages into e-wrks org, then reproducing these issues will be more practical.

[Avi-D-coder]

Try the new version, empty lists should be fixed now.

[complyue]

Still spitting unexpected entries:

    - path: "./e-wrks/sedh/host.hs/gwd/Main.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/gwd/default-language:.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/gwd/Haskell2010.hs"
      component: "sedh:exe:gwd"

    - path: "./e-wrks/sedh/host.hs/forage/Main.hs"
      component: "sedh:exe:forage"

    - path: "./e-wrks/sedh/host.hs/forage/default-language:.hs"
      component: "sedh:exe:forage"

    - path: "./e-wrks/sedh/host.hs/forage/Haskell2010.hs"
      component: "sedh:exe:forage"

    - path: "./e-wrks/sedh/host.hs/src"
      component: "sedh:lib"

Btw, how you triage the absolute package path situation? Is it worth and easy enough to support absolute paths?

[Avi-D-coder]

I fixed #30 so, please try the latest release from hackage or master.

[complyue]

verified for master, the output are all expected now 👍