Automation Engineer - Worlwide Remote

Automation Engineer - Worlwide Remote at Invisible Technologies in United States.

Responsibilities

• Create crawlers for each source URL by using Python modules [scrapy, selenium, requests, BeautifulSoup and splash]
• Create and maintain scrapy pipelines and middlewares to manage the output from the crawlers.
• Create crawlers for all types of websites irrespective of the technical roadblocks.
• Manage the crawlers to overcome technical challenges like IP ban, geolocation ban, captcha and bot blocking services.
• Write SQL queries to manage database operations using Python modules like sqlalchemyDeploy the python scripts / crawlers to Linux based AWS servers

Requirements

SkillsMust Haves

Strong hands-on experience in Python programming

• Good experience with scraping libraries such as Requests, BeautifulSoup, Selenium and Scrapy.
• Proven 3+ experience working on web development frameworks such as Flask/Django/FastAPI/Tornado/PandasKnowledge of building APIs and services using REST.
• Experience with any RDBMS and strong SQL knowledge.
• Clear with Object-oriented concepts.
• Excellent troubleshooting skills.
• Proficient understanding of code versioning tools like git.
• Basic understanding of front-end technologies, such as JavaScript, HTML5 and CSS3Understanding of fundamental design principles behind a scalable application.

Nice to have

• Familiar with UI frameworks like Angular/ReactJS
• Strong unit test and debugging skills.
• Strong knowledge of data engineering platforms such as Airflow.
• Experience with the Google CloudDockerAgile and Scrum. Even better if have worked in Jira and confluence.
• Experience with ETLCeleryExcellent interpersonal skills and ability to to work with a diverse team
• Required: Technical Skills
• Python
• JavaScript
• PostgresSQL
• Web Scraping using Requests, BeautifulSoup, Selenium and Scrapy
• Frameworks such as Flask/Django/FastAPI/Tornado/Pandas
• Building APIs and services using REST

Invisible Technologies offers unusual services (a combination of outsourcing and automation) to fast-scaling, innovative companies. Each company understands and uses our services in a variety of ways. It’s critical we can communicate to segments of users with visually appealing, and highly communicative assets. You’re helping us explain the benefits of our product in a way that words alone never could.