Enterprise level, full stack web scraping suite for business
4.7/5 (28 avaliações)Vantagens
It really saves my time as a web scraping specialists. I always need rotated proxies for my client needs. I keep creating a new project and it is a lot of hassle of I have to buy proxies, renew proxies, and create proxy rotation on my own. I am glad I don't even need to think about it. It just works!
Desvantagens
Well, it's just doesn't have enough advanced documentation on Scrapy Splash combined with Crawlera. Their development team in Github is very responsive though.
Our company, whatoplay.com, aggregates data from multiple sources around the world. Since we started using Crawlera, we haven't really encountered big issues. Our common ones in the last 6 months is just reaching our limit, so we just had to increase our plan.
Vantagens
We're only using Crawlera for our data scraping needs and it has worked seamlessly since we started using it in 2016. When we had to upgrade to a higher limit, the transition was fast, and no dev time was wasted updating our existing codebase.
Desvantagens
Pricing can be a bit tricky especially if your need is right in between the plans.
I have a good experience with scrappinhumb, in a matter of minutes I can develop and publish a new spider.
Vantagens
The software delivers the exactly promised result. It has a friendly interface and an efficient support.
Desvantagens
You need a roadmap of new features and the pricing of third-party services.
Thank you for the review. All new features will be announced in our Support Center.(https://support.scrapinghub.com/support/home) We also have an Ideas Forum (https://support.scrapinghub.com/support/discussions/forums/22000200101) where we would love your input on ideas or new features you would like to see. We appreciate all input from our Customers to continue to help us improve our products.
I use and recommend that platform for years for my customers which need production-ready enterprise-grade data scraping systems.
Vantagens
- Original and flexible technology
- No vendor-lock
- Easy to use for professionals
- Pretty convenient API system for integration with third-party solutions
Desvantagens
- Not so easy to use by non-professional IT person that still wants to use data scraping
- Lack of ability to create some kind of simple and clear user interface for such a persons
- No simple solution for distributed/high-volume crawling
- Lack of monitoring and alerting, non convenient logging system
- Overpriced Crawlera
Thank you for your review we appreciate all customer feedback. We do have a managed service offering for non IT professional who want access to the Data without having any scraping skills.
Started using it 2 years ago, learning curve totally rely on the framework because the cloud platform is pretty intuitive, self explanatory and easy to use.
Whenever I needed support I have been response fast and with a favorable solution.
Vantagens
A full integrated platform for a framework well done for it purpose. Easy to use, based on others frameworks structure, so if you are used to web development in python then it's a piece of cake to create spiders.
The integration (scrapy + scrapinghub) its really good, from a simple deployment through a library or a docker makes it suitable for any need.
Good support and constant improvement on the platform.
A lot of plugins and, open to any feature needed.
Desvantagens
So far there is nothing I dislike about it.
Never had a problem, and customer services always replied to our requests promptly, and with attention to detail.
Vantagens
We are very happy with Crawlera. It works like a charm with our Scrapy projects, just adding a few lines of code we completely forgot about proxies that ended up failing anyway.
Desvantagens
Some requests end up being very slow, but good news is most of them end up going through.
Portia aside, I highly recommend this platform. It's cheap and works really well. If you know how to use Scrapy, this is easier than launching a bunch of VMs, etc. and relatively inexpensive.
I knock Portia because I wanted it to work, and it will with A LOT of patience, but, if you know your way around Python and Scrapy, this is the scraping cloud platform for you!
Again, there technical support has been great for me. Most of my issues have been stupid mistakes by me, but they are happy to help in a reasonable time frame (usually under 24 hours/ under 8 hours in my experience).
Vantagens
- Works perfectly as long as you do not need Portia
- Fast
- Many available features in one place
- It fulfills my scraping needs well
- EXCELLENT technical support
Desvantagens
- Portia (beta) is almost unusable. I got a couple very basic programs out of it before I taught myself how to use Scrapy. Still, if you are looking for a value and have time to deal with the lag/hiccups in Portia, it is by far the best value for a simple point and click scrapying platform that I found. You can make it work, you will just want to put your head through the screen before you are done...
Scrapinghub has enabled us to streamline the scraping process for our company and sets the foundation for future scaling. I like the fact that scrapyhub handles the "Ops" part of scraping like provisioning cloud resources, staging data, and rotating IP addresses.
Vantagens
Scrapy is amazing, python api is easy to use, job dashboard is easy to understand, good documentation and code.
Desvantagens
Dynamic pages sometimes hard to scrape, nothing wrong with lua (for splash) as a language, but can be minor barrier to entry for python/javascript users.
I am very happy with Scrapinghub. Whenever I need to run a small scraper that I can't run on my laptop (since I don't leave it turned on 24/7), I just run it on Scrapy Cloud. Meanwhile, Crawlera is just the best!
Vantagens
I have been using Scrapy and Scrapinghub's services since 2013 and I'm so far very satisfied with their services. Crawlera, their proxy service, works very well! I don't have to setup a proxy farm anymore or configure my scrapers to point to thousands of proxy services as they do all the grunt work for you (it's all automated).
Desvantagens
Minor hiccups from time to time on the dashboard. It only affects me when I want to see historical stats but this problem doesn't affect functionality.
Scrapinghub gives my business higher quality raw data so my clients get better results.
Vantagens
This software allows me to access 97% of the web resources I need. Other proxy servers I have used gave me 75% maximum.
Desvantagens
That its a time based subscription. I would prefer being able to buy a certain number of requests and renew when those run out.
Thanks you for your review and input on our pricing model - we appreciate all customer feedback and have fed this into our product team for review.
The best out there for any kind of scraping need. They new AI based scraping api is a game changer
Vantagens
The best out there for any kind of scraping need. They new AI based scraping api is a game changer
Desvantagens
Nothing bad about it. Beats the competition out of the water
Coming from a "legacy" environment where everything was build from scratch I have to say that my experience with Scrapinghub is really positive. I started using some more advanced features such as ItemLoaders, Middleware etc.
Vantagens
Scrapinghub is so easy to use. With few setups you're ready to build your first spider. The integration with github and other addons (such as crawlera) makes things even easier to manage code deploy and proxy network management. Support for Python 3 is also a great improvement
Desvantagens
I'm still in early days with scrapinghub and at the moment did not see any big issues to take into account. One things that comes in my mind to be improved is the documentation.
Thank you for your review we appreciate all customer feedback and are constantly looking to have better documentation with continuous improvements underway. If you see any issues in our current documentation we would love to hear from you. Please submit via our Support Center.(https://support.scrapinghub.com/support/home)
Vantagens
it helps a lot to avoid the self-hosted spiders, this is the primary reason I like it.
and I know it is developing to make it more ease to use, like github integration, however I have set up the CI tools to publish the spider automatically for every code change, so I don't use it.
Desvantagens
however it is always a pain that it is more difficult to bypass the bot detection, especially for big website, like amazon, and the other websites protected by Distil. I know this is not the fault of scrapinghub, but it is a big pain. and I know you have a service Crawlera for this purpose, but it is quite expensive.
Thank you for your review we appreciate all customer feedback. We agree and antibot research is something we are also focused on to help our customers.
Vantagens
I use the crawlera proxy system from Scarpinghub. It is very straightforward to use and results are great.
Desvantagens
No cons until now. Once in a while a server/IP gets banned but they have a big pool of IPs to serve you.
Vantagens
I use the Python scrapy module to write crawlers to monitor competitors prices. The integration with Github makes it real easy to deploy code to the hub. The dashboards are very useful to monitor progress and schedule jobs. Really happy with this offering.
Desvantagens
In the beginning it was somewhat cumbersome to automate data extraction. Once I started using Python scrapinghub module life became easy. So also a pro!
From what I've used (basic spiders) it has been great, the page is really easy to use and the cli makes deploying easy.
Vantagens
Open source, well documented and very efficient.
Desvantagens
Splash needs to be migrated to another tool IMO. I don't find LUA easy to work with as a developer; not because of the language but debugging and maintaining code is really hard. If you use scrapy-splash there is a black box that cannot interact with. I think going mainstream with javascript or python whilst unifying browser scrapping and scrapy itself would be an amazing decision.
So, overall I would love to see a merge between scrapy and a browser scraper.
Crawlera is a great platform for crawler. I used it for 2 years in my business and jobs.
Vantagens
Your reliability an trust about proxy ip list.
Desvantagens
You payment method just accept international credit cards. I used it from Brazil.
Good, they are responsive and professional. Would recommend.
Vantagens
As a business once you hit a certain size it makes sense outsourcing webscraping. Scrapinghub has the required specialism to do a good job.
Desvantagens
Still needs involved work from the client side to ensure data quality and consistency.
Thank you for your review we appreciate all customer feedback. We agree and data quality is very important to us as well. We are constantly looking to improving our QA framework and working with our customers to ensure consistent data quality.
Vantagens
Very easy to use. Great dashboards for monitoring jobs. Very competitive pricing. Friendly customer service.
Desvantagens
There's very little to complain about with Scrapinghub - it's genuinely a really great product!
Vantagens
Crawlera makes it really easy for me to process thousands of requests without worrying about rate-limits. I love that I don't have to worry about retry policies or IP blacklisting because Crawlera takes care of that for me! Plus, by charging by requests instead of bandwidth, it's much easier for me to estimate my expenses based on my needs.
Desvantagens
I don't like that the usage numbers are delayed by so many hours.
The pricing is also a little high. I wish there was some way to bring that down or get more value into the plans.
Vantagens
The effectiveness - it simply works better and is more consistent than other proxies.
Desvantagens
The monthly limit on requests and how there is no intermediary plan between C10 and C50.
I came to scrapinghub as a young ambitious analyst. The free to use service is what kept me. When more scraping projects comissioned I contacted other providers to find out who would deliver the most value. There really was no alternative: the best prices, virtually unlimited data, and A+ team at scrapinghub.
Vantagens
Simplicity, no-bullsh*t services, modular services & addons, very low service downtime. I can keep going and still won't do these guys justice
Desvantagens
Payment platform wasn't a walk in the park. Guides and knowledge base are not outdated but can use a 2019 refresh
I am solving meta search price problems within the cannabis industry.
Vantagens
It was very easy to start creating crawlers and launching them on scrapinghub. Plug and play. Ready to start launching a fleet of crawlers to move my project to the next level.
Desvantagens
The price was a bit hefty for running multiple crawlers.
Vantagens
· Scraping situation is easy to see visually
· Can be finely implemented such as time to activate scraping
· Ease of deployment
Desvantagens
· I try to deploy from github, but I can select only my own repository (Can not deploy from organization's repository?
would recommend anyone who web scraps website data for mining intelligence.
Vantagens
we have used scrapinghub for generating proxy ip's to mine company web data and career page information. has been very useful in assisting us to stand up our own skill vertical search engine.
Desvantagens
needs a lot of customization if the scripts run into issues. overall a good product for its use cases
It makes me work really fast but still secure for my clients. I look really professional even though what I did was only suggesting and installing Crawlera. It's awesome!