What are people doing about the gazzillion file problem?

I was working on creating a new admin section in Discourse.

It is used to manage groups… so off I go.

I edit the routes/admin_routes.js and add a file.

I then create routes/admin_groups_route.js and add another file.

then I create admin/models/group.js model, and admin/views/admin_groups_view.js and admin/views/admin_groups_controller.js

finally I create admin/templates/groups.handlebars.js

So, creating this bit of functionality required 5 new files and 1 file change. As I am developing I am constantly jumping between files, losing my place and having to reopen and explore.

As it stands Discourse now serves over 370 files in debug, something that takes upwards of 4 seconds (sure you can circumvent and shorten it, but then you take a hit on debugability)


Are people following the one file per class rule with Ember, or are some amalgamating files? Personally I would be a lot less confused having a single admin/groups.js file that encomposes everything (at least for smallish bits) and then a single template file. So much less jumping around and hitting find in file or ninja navigation required.

2 Likes

I have one main CoffeeScript file that imports each of my other files (organised into the correct category).

When I’m quickly trying things out I just add to the bottom of that file, but then once things are starting to work (and I’ve made sure I know which objects I actually need to generate…) I move them each into their own modules.

Kind of a mix - one file for messy development, clean up later.

1 Like

I started by doing something similar to what you describe, but I was quickly turned off my the number of files. My current rule of thumb is to combine files that achieve a modular or similar feature, splitting them out when it starts to get visually painful.

Not an exact science, I know :smile:

1 Like

I’ve been experiencing this same issue, Sam. What I’ve started to do (with routes and controllers that I know won’t contain a large amount of code) is combine resources into the same file. For instance, I may have an edit, new and show route under a ‘Users’ resource. I used to have a separate route file for each of these, but now I’m creating one ‘Users’ route file and instantiating all of the routes I need within it. This seems to work fairly well so far.

2 Likes

I like this approach. When having one file isn’t so beneficial anymore, it’s time to split it into multiple files. How do you organize and name these all-in-one files?

One problem is that people will need to know to look in these files that have many classes, as well as the usual one-class-per-file directories.

I’m using requirejs. I have a hierarchy of dependencies.

ConfigureApp
  App
  rt/IndexRoute
  rt/FoodsIndexRoute

App
  Router
  Store

rt/IndexRoute
  ctl/IndexController
  vw/IndexView

rt/FoodsIndexRoute
  vw/FoodsIndexView

vw/ApplicationView
  tmpl/applicationTemplate

vw/IndexView
  vw/ApplicationView
  tmpl/indexTemplate

vw/FoodsIndexView
  ctl/FoodsIndexController
  mdl/Food
  vw/ApplicationView
  tmpl/foodsIndexTemplate

ctl/FoodsIndexController
  mdl/Food

The “ConfigureApp” module requires the App module, which requires the Store and the Router. And so on. The modules each have side effects to attach their object to the App. (They all depend on App as well as Ember and in some cases ember-data.

I like(d) this arrangement because it felt like I wan’t “required” to maintain a laundry list of objects/classes. However there is a serious problem here: the background-loading nature of requirejs. Ember happily routes on the initial page before all of the modules are loaded.

For now, I’ve solved this by requiring everything in ConfigureApp (though I have left the dependencies intact). I’m using deferReadiness/advanceReadiness and I know I could use them in each module to hold up the entire initialization at the ConfigureApp level. But I still want to crack this. I want to benefit from being able to dynamically load each dependency, but to wait on any route that doesn’t have all of the ingredients in place. Obviously, the intention here is to get faster initial load time.

What I’m looking for is an appropriate hook in the Router that will allow me to know when a particular extant route is actually needed. In that case, I’d like to stall the Router until its dependencies are all in place.

Perhaps someone could point me to such a hook? I’m just getting started with Ember so I’m not at all familiar with hooks that don’t receive coverage in the guides. It needs to be a place where I can know which route the Router has chosen, and where I can know whether such a route exists (as opposed to an intentionally undefined one).

I’m building an app with each class in a separate file, organized by category (models, controllers, etc) instead of per resources. My app hasn’t grown large enough yet (~50 js files and 17 hbs files ) so I’m bundling in a single file; but not bundling or minifying in dev due to debug.

I’m doing it in .NET using compiler directives, so right now I have something like this:

#if debug
    bundles.Add(new ScriptBundle("~/bundles/name").Include(
        "~/scripts/directory/*.js",
        "~/scripts/other_directory/*.js"
    ));
    // other bundles here
#else
    bundles.Add(new Bundle("~/bundles/name", new JsMinify()).Include(
         "~/scripts/directory/*.js",
         "~/scripts/other_directory/*.js"
    ));
    // other bundles here
#endif

as for directory structure, I currently have something like this:

~/
 |_scripts
     |_app
     |   |
     |   |_controllers
     |   |    |_ *.js
     |   |
     |   |_views
     |   |    |_ *.js
     |   |
     |   |_models
     |   |    |_ *.js
     |   |
     |   |_setup
     |   |    |_ *.js (init external libs and helpers)
     |   |
     |   |_routes
     |   |    |_ *.js
     |   |
     |   |_templates
     |   |    |_ *.hbs
     |   |
     |   |_app.js
     |
     |_ *.js (this would be "vendor" scripts)

I’m still working through it, so this might eventually change.

2 Likes

One very important update here is that I hacked up Discourse to support debugging with the very large amount of files we have:

https://github.com/discourse/discourse/blob/master/lib/middleware/turbo_dev.rb

See also:

https://github.com/rails/rails/issues/10291

3 Likes

I actually like having many files because it is faster for me to switch between them by using cmd-T and typing part of the class name than searching within a file, or switching and searching.

Rails has many files too, I think the main difference is we use one file per route rather than in one place? The difference of course is our routes are classes, not just one liners.

Either way I’d be happy to join together all our routes. I think having all stuff related to a group in one file would be a bit overkill though.

Control-P, split panes, and buffers in vim seem to keep me plenty happy.

Time for EmberMine? :stuck_out_tongue:

1 Like

I personally like having lots of small, expressive files that I can easily grok. Typically you don’t need to be looking at more than the model, controller, and template at any given time. I use CtrlP plus two or three buffers open at the same time and have not experienced a problem, either with Rails or Ember, which both have lots of small objects with single responsibilities.

If the lots of small files is a “best practice” thing, someone is going to have to exert some pressure on https://github.com/rails/rails/issues/10291 , as it stands the defaults in rails (even version 4) are unfriendly for large number of tiny files.

As it stands josh does not want this touched until source maps are fully out there, but historically rails has been very slow to adopt new revs of sprockets, so it may be years out.

2 Likes

@sam Does your middleware hack still work if you are using javascript/css files that need to run through a pre-compiler such as coffeescript?

sure does, works fine either way that’s the beauty of it

Awesome, I’ve just implemented it in one of my apps and it has cut down the refresh time considerably. Thanks!

One thing that we’ve had luck with is using something like https://github.com/wycats/minispade to register each file as a logically separate entity but coalesced into a single file. If you append a sourceURL annotation to the end of each one, the Chrome and Firefox debuggers treat them like separate files.

4 Likes

If you are using grunt you can do something similar with grunt-neuter using the includeSourceURL option.

2 Likes

Theres an RFC somewhere that talks about grouping the files?