Skip to content
This repository has been archived by the owner on Aug 11, 2022. It is now read-only.

npm publish missing index.js #5082

Closed
pgilad opened this issue Apr 15, 2014 · 129 comments
Closed

npm publish missing index.js #5082

pgilad opened this issue Apr 15, 2014 · 129 comments

Comments

@pgilad
Copy link

pgilad commented Apr 15, 2014

Happened to me twice already, while publishing a package I wrote (https://github.com/pgilad/gulp-angular-htmlify if it matters), the package was published without index.js.

I had no notice or info that anything went wrong during npm publish phase. But then got notified by a user that the package wasn't installing with index.js.

The only thing that I can think of that caused this, other than a bug, is that I had the project open in Vim, that may have caused a failure to publish the file.

Did anyone run into a similar case? Is there anyway to verify the package contents other than installing it as a dependency and seeing what's installed?

This happened with:
git version 1.8.3.2
npm version 1.4.6
node version 0.10.26
ubuntu 13.10

@isaacs
Copy link
Contributor

isaacs commented Apr 16, 2014

I'm not sure what's going on there. Using npm version 1.4.7, I'm seeing it include the index.js when it creates a tarball from that git repo.

[gh:npm/servers/master]isaacs@urdva:~/dev/npm/servers v0.10.27-pre
$ npm cache add git+https://github.com/pgilad/gulp-angular-htmlify.git

[gh:npm/servers/master]isaacs@urdva:~/dev/npm/servers v0.10.27-pre
$ npm cache ls gulp-angular-htmlify
~/.npm/gulp-angular-htmlify
~/.npm/gulp-angular-htmlify/0.0.8/
~/.npm/gulp-angular-htmlify/0.0.8/package/
~/.npm/gulp-angular-htmlify/0.0.8/package.tgz
~/.npm/gulp-angular-htmlify/0.0.8/package/.editorconfig
~/.npm/gulp-angular-htmlify/0.0.8/package/.gitattributes
~/.npm/gulp-angular-htmlify/0.0.8/package/.gitignore
~/.npm/gulp-angular-htmlify/0.0.8/package/.jshintrc
~/.npm/gulp-angular-htmlify/0.0.8/package/.npmignore
~/.npm/gulp-angular-htmlify/0.0.8/package/.travis.yml
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-basic.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-complex.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-custom.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-templates.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/noangular.html
~/.npm/gulp-angular-htmlify/0.0.8/package/index.js
~/.npm/gulp-angular-htmlify/0.0.8/package/license.md
~/.npm/gulp-angular-htmlify/0.0.8/package/package.json
~/.npm/gulp-angular-htmlify/0.0.8/package/readme.md
~/.npm/gulp-angular-htmlify/0.0.8/package/test.js

Can you provide any other details? If you do npm pack && tar ztvf gulp-angular-htmlify-0.0.8.tgz, what does it output?

@pgilad
Copy link
Author

pgilad commented Apr 17, 2014

Hi this is something inconsistent that happened twice (on 2 version releases).
It happened on v0.0.6, and when I do:

$ npm cache add gulp-angular-htmlify@0.0.6
npm http GET https://registry.npmjs.org/gulp-angular-htmlify/0.0.6
npm http 200 https://registry.npmjs.org/gulp-angular-htmlify/0.0.6
npm http GET https://registry.npmjs.org/gulp-angular-htmlify/-/gulp-angular-htmlify-0.0.6.tgz
npm http 200 https://registry.npmjs.org/gulp-angular-htmlify/-/gulp-angular-htmlify-0.0.6.tgz

$ npm cache ls gulp-angular-htmlify
~\AppData\Roaming\npm-cache\gulp-angular-htmlify
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\.cache.json
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package.tgz
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\package.json
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\readme.md

So you see it's missing index.js for some reason, and has an unusal .cache.json file. What other details can I provide?

If I run your command I see:

npm pack && tar ztvf gulp-angular-htmlify-0.0.8.tgz
gulp-angular-htmlify-0.0.8.tgz
tar: Record size = 18 blocks
-rw-rw-rw- 0/0             883 2014-04-17 11:00 package/package.json
-rw-rw-rw- 0/0            2094 2014-04-17 11:00 package/index.js
-rw-rw-rw- 0/0            2964 2014-04-17 11:00 package/readme.md

@pgilad
Copy link
Author

pgilad commented Apr 24, 2014

Hi @isaacs this happened to me again on a different project I'm working on, perhaps this is machine related (Ubuntu 13.04 + 14)

I ran the following commands:

[master][~/repos/gulp-sitemap]$ npm publish
npm http PUT https://registry.npmjs.org/gulp-sitemap
npm http 201 https://registry.npmjs.org/gulp-sitemap
+ gulp-sitemap@0.2.0
[master][~/repos/gulp-sitemap]$ npm cache add gulp-sitemap@0.2.0
npm http GET https://registry.npmjs.org/gulp-sitemap
npm http 200 https://registry.npmjs.org/gulp-sitemap
npm http GET https://registry.npmjs.org/gulp-sitemap/-/gulp-sitemap-0.2.0.tgz
npm http 200 https://registry.npmjs.org/gulp-sitemap/-/gulp-sitemap-0.2.0.tgz
[master][~/repos/gulp-sitemap]$ npm cache ls gulp-sitemap
~/.npm/gulp-sitemap
~/.npm/gulp-sitemap/.cache.json
~/.npm/gulp-sitemap/0.2.0/
~/.npm/gulp-sitemap/0.2.0/package/
~/.npm/gulp-sitemap/0.2.0/package.tgz
~/.npm/gulp-sitemap/0.2.0/package/.npmignore
~/.npm/gulp-sitemap/0.2.0/package/license.md
~/.npm/gulp-sitemap/0.2.0/package/package.json
~/.npm/gulp-sitemap/0.2.0/package/readme.md

As you can see, for some reason it packed .npmignore and it didn't include index.js. I had to republish another version.

Any clue on this?

@hiddentao
Copy link

Seeing something similar with with my module - gulp-bench. I tried deleting my .npmignore prior to npm pack, all to no avail.

@hiddentao
Copy link

In my case the problem was this line in package.json:

files: ['index.js']

Duh. To get it working I changed it to:

main: 'index.js'

@pascalduez
Copy link

Same issue here happening on various projects. (i.e: grunt-sassdoc, gulp-sassdoc)

The problem is this issue is not consistent, sometimes it's fine, publishing the package with the right files, but sometimes it randomly exclude some. Often the index.js but I've also had the case with other files, like Readme and co.

I also switched from files: ['index.js'] to main: 'index.js', it makes no difference.

node v0.10.30
npm v1.4.23
OSX

wmluke added a commit to wmluke/gulp-inline-angular-templates that referenced this issue Sep 7, 2014
* Removed `files` field from package.json
* Added `npmignore`
@othiym23
Copy link
Contributor

Is this issue still occurring with npm@2.0.0?

@pascalduez
Copy link

Is this issue still occurring with npm@2.0.0?

Hard to tell at that point, since it's happening kind of randomly.
I'll report back whenever it happens.

@othiym23
Copy link
Contributor

OK. I'm going to close this for now, because I believe it's been addressed by one of the many fixes that we've made around race conditions. If it happens again, open a new ticket. Even better would be if you can include a link to a gist with the output of npm publish -ddd for a bad publish. Thanks for your patience and help!

@othiym23 othiym23 added the bug label Sep 21, 2014
@ghost
Copy link

ghost commented Sep 26, 2014

I'm getting this issue every other npm publish. I'm on npm version 2.0.2.

@othiym23
Copy link
Contributor

@substack thanks for finding this issue. I'm going to reopen and ask that if anyone on this thread can put together a repro case for this issue, they should give me a shout. This bites James hard because he publishes so many packages, but it's a serious issue to ever be happening at all, and I want to fix it!

@othiym23 othiym23 reopened this Sep 26, 2014
@othiym23
Copy link
Contributor

@substack does npm ls or @thlorenz's irish-pub show anything weird on any of the packages that have had this problem? Can you get it to happen just running npm pack?

@othiym23 othiym23 added this to the 2.1.4 milestone Oct 4, 2014
@othiym23 othiym23 removed this from the 2.1.4 milestone Oct 7, 2014
@pgilad
Copy link
Author

pgilad commented Oct 19, 2014

I haven't encountered this bug for a very long time (perhaps since reported). On another note - I don't use .npmignore anymore, but use the files array in package.json.

@othiym23
Copy link
Contributor

Thanks, @pgilad. @substack, when was the last time you saw this issue? I'm hesitant to close it, because I still don't know what could have been / was causing it, but it's lower priority if it's not currently affecting anyone.

@pascalduez
Copy link

when was the last time you saw this issue?

Well, I encountered it like a week ago.

Still using .npmignore.

@othiym23
Copy link
Contributor

@pascalduez with which version of npm?

@pascalduez
Copy link

with which version of npm?

I guess it was 2.1.3 back then.

@jbeckton
Copy link

using npm pack and having this issue my index.js file is not included in the package

@othiym23
Copy link
Contributor

@jbeckton what does npm -v return?

@jbeckton
Copy link

@othiym23 3.9.5

@jbeckton
Copy link

updated to the current version and the problem went away.

@busches
Copy link

busches commented Sep 22, 2016

Good to close then?

On Wed, Sep 21, 2016 at 6:14 PM, Jesse Beckton notifications@github.com
wrote:

updated to the current version and the problem went away.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#5082 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA0WZOtZ6m_kRre7xOwPrk-ZDVsgWfyIks5qsbpLgaJpZM4Bywuj
.

@jbeckton
Copy link

yes.. Thanks

@busches
Copy link

busches commented Sep 22, 2016

@othiym23 how's the progress on opening the tracking issues?

@SamVerschueren
Copy link

Update npm.

@CYBAI
Copy link

CYBAI commented Nov 8, 2016

@SamVerschueren sorry for that I delete the comments and re-describe my problem.

I have to npm install with git remote url and it will install dependencies successfully.
However, it won't run prepublish scripts like npm install <package> do, so I cannot require my module due to no index.js there.

here's my package.json and .gitignore

package.json

{
  ...
  "scripts": {
    "compile": "babel-cli src --out-dir lib",
    "prepublish": "npm run compile"
  },
  "files": [
    "lib/"
  ],
  ...
}

.gitignore

lib
node_modules

@ghost
Copy link

ghost commented Dec 7, 2016

I am still having a problem with randomly missing files in the tarball created by npm pack or npm publish. This is the only issue I can find which appears to be related. My npm version is

$ npm -v
3.10.8

This the one which currently ships with Node.js. I haven't ever used a version other than the one that comes with Node.js. At some point in the last few months, an update to Node.js has shipped with a newer npm which randomly omits a file from tarballs. I'm not exactly sure when this happened, because we don't discover it until deploying the package to a server.

There's a second perhaps unrelated issue too, which is npm can crash creating a tarball:

$ npm pack
npm ERR! cb() never called!

Generally though, in the specific case I am concerned about, the problem is that the tarball either contains the complete 206 files, or just 205. It appears to be about as random as a coin toss. So I tested by doing npm pack 32 times (30 actual results plus crashes) and the outcome was

  • 2 complete failures (npm ERR! cb() never called!)
  • 18 apparent success but with only 205 files
  • 12 success with 206 files

It's always the same file that's missing. When it fails, one particular subdirectory in the tarball looks like this:

package/dealercam/play.js       4195
package/dealercam/upload.js     8885

It should look like this:

package/dealercam/play.js       4195
package/dealercam/transcode.js  3974
package/dealercam/upload.js     8885

So it's always the same file that's missing.

Unfortunately, this breaks our automated mechanisms for deployment and turns it into a manual process. Automated deployment was just crashing servers, and while doing npm unpublish and npm publish a second time generally fixes the problem, it's really not ideal!

Apologies if this is not the right place for this, but it seems like it might be the same issue.

@laughinghan
Copy link
Contributor

@steve-p-com I can't speak for NPM but that sounds to me like just enough detail to make a compelling case that your issue is relevant to this, but not enough for them to actually help you.

  • Can you still reproduce? (If not, what if you use the same machine, and check out that same commit of your project, and revert to the version of Node and npm you were using then (if that's changed)?)
  • Was that on a build server or on your local dev machine? If server, can you repro locally? If local, can you repro on a coworker's dev machine?
  • Can you repro on multiple OSes? If not, what if, on the non-reproing OS, you're inside a VM running the repro-ing OS?
  • Assuming this is closed source so you can't just provide a GitHub link, have you had any luck trying to pare down your test case to produce a minimal one?

I'm a little anxious to have this fixed once and for all, it hasn't been an issue for me since the fstream fix but I'd love to be able to let my guard down!

@laughinghan
Copy link
Contributor

@CYBAI that sounds like definitely not relevant to this issue, you should open a separate issue. (I'm not sure it's a bug at all, what about the install hook? Please don't reply in this thread, just tag me in the separate issue.)

@ghost
Copy link

ghost commented Jan 3, 2017

@laughinghan I have been on vacation since I posted that message 27 days ago so I haven't taken any further steps. I am likely to be quite busy in the short term so may not get back to looking at it immediately. This was on a local dev machine which is Windows 7 64 bit. It's the only place I publish from, but in the medium term future we will be using a (linux) build server, at which point I'm really hoping the problem will go away.

@youfoundron
Copy link

youfoundron commented May 8, 2017

My Mistake (and it's an easy one to make)

Was having this issue and this post cleared it up for me.

My build directory was in my .gitignore, npm defaults to .gitignore if no .npmignore file exists. Created an empty .npmignore to resolve.

@npm-robot
Copy link

We're closing this issue as it has gone thirty days without activity. In our experience if an issue has gone thirty days without any activity then it's unlikely to be addressed. In the case of bug reports, often the underlying issue will be addressed but finding related issues is quite difficult and often incomplete.

If this was a bug report and it is still relevant then we encourage you to open it again as a new issue. If this was a feature request then you should feel free to open it again, or even better open a PR.

For more information about our new issue aging policies and why we've instituted them please see our blog post.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.