3👍
I’m answering my own question because I discovered what the problem was. Heroku for some reason was not able to find scrapy
when I executed the command from a sub-directory and not the top-level directory.
The command heroku run ...
is generally run from the top-level directory. For my project which uses scrapy, I was required to go to a sub-directory and run the scrapy
command from the sub-directory (this is how scrapy is designed). This wasn’t working in Heroku. So I went to the Heroku bash by typing heroku run bash
to see what was going on. When I ran the scrapy
command from the top-level directory, Heroku recognized the command but when I went to a sub-directory, it failed to recognize the scrapy
command. I suppose there is some problem related to the path. From the sub-directory, I had to specify the complete path to scrapy
(~/bin/scrapy crawl spidername
) to be able to execute it.
To run the scrapy
command without going to the Heroku bash manually each time, my work around this problem was that I created a shell script containing the following code and put it under the bin directory of my top-level directory and pushed the changes to Heroku.
bin/scrapy.sh :
#!/usr/bin/env bash
cd ~/project/spiderSubDirectory
~/bin/scrapy $@
After this was done, I could execute $ heroku run scrapy.sh crawl spidername
from my local bash. I suppose its not the best solution but this works.
0👍
Isn’t the way external commands are run in Heroku like – heroku run
appdir command?
It’s actually heroku run command
. By including your appdir in there, it resulted in an invalid command. Heroku’s output doesn’t give useful error messages when these commands fail, and instead just tells you that the command finished which is what you’re seeing. So for you, just change the command to something like:
heroku run scrapy crawl spidername
- [Django]-C# vs Python: XML Handling/Processing Productivity
- [Django]-Error when installing mysqlclient
- [Django]-Django – Models – Recursively retrieve parents of a leaf node
- [Django]-Email not translated inside a Celery Task