Website crawler perl trabalhos

I need an experienced and proficient website crawler who can help me extract Text content from Database Tables on my website. Key requirements: • Acing in finding, fetching, and structuring data. • Proficient in delivering the retrieved data in Plain text format. • Emphasis on fetching Text content from the Database Table, thus understanding of database structures is a must. Ideal skills include web scraping, data mining, and database understanding. It would be best if you had an eye for detail as accuracy is key in this project. With your expertise, you should be able to navigate through complex website layers to get to the desired content.

I'm in need of a skilled freelancer who has expertise in Perl scripting, primarily for file manipulation tasks. Key Requirements: --Gather date from a table in oracle. Based on that date find the records available in a home directory and store it in a .txt file. Skills and Experience: - Proficiency in Perl scripting is a must. - Experience in handling various file formats, specifically .txt and .csv. - Strong understanding of file operations including renaming, formatting, and sorting (although the specific task was skipped). I need the code done within 2 days and willing to pay 3000 which would be double the cost of what is given .

We are in need of a knowledgeable and skillful programmer who can edit on GoLang code. The main task involves Ethereum network crawling. We already have a running code to crawling the Ethereum network. However, we still need to do some changes on GoLang code (designing crawler), such as partition the ID_space into sections, and crawl using multiple crawlers in Ethereum not only single crawler as what we have. In addition, there might be more few changes on the code. The ideal candidate will: - Understand DHT-based network and Kademlia - Knows a little about Ethereum (blockchain) - Have experience in GoLang - Possess a deep understanding of DHT network crawling (p2p) - Have previous experience with crawling networks Your responsibility will be building on top of running code...

I'm currently using PERL 5.10 system and requires a proficient expert to aid in modifying how passwords are hashed in my system. Initially, we employed SHA-256 but we're looking to switch to PBKDF2. Ideal Skills and Experience: - Proficient in PERL 5.10 - Familiarity with SHA-256 and PBKDF2 hashing methods - Prior experience in password re-hashing Task Objectives: - Identify current SHA-256 hashing method in the system - Modify hashing method to use PBKDF2 instead - Validate successful implementation of the PBKDF2 hashing method The project necessitates a strict focus on security and confidentiality given its sensitive nature.

Script Feature: - Login to a list of SSH servers by domain or IP (from text file) - Use a set of usernames + password (can be multiple passwords) to login to the servers. - Run Bash Script on the connected server (which comes from a txt file on the server running the script, not on the connected to server) - Timeout if SSH server did not answer. - Display results at the end of a run (show which servers the script did not run because of errors). Login simultaneously to all available ssh servers. Not server by server.

I need to convert the following Perl code to PHP: The expected output is a PHP class that can convert a docx file into text. The attached zip contains a mockup of such class (with demo code) and a test docx file. The task is straight forward. If you place a bid, expect it to be accepted. If you fail to deliver, you'll be subjected to a bad review. This is to avoid wasting my time with autoposters and people who lowball the bid and then try to ask for more money. You have been warned. Read the brief, check the code and place a bid accordingly.

Google Jobs Scraper project requiring skills in .NET, Python, Selenium and SQL. The task is to create a crawler to extract job listings based on predefined queries, manage the data in a structured manner, and provide an interface for controlling and monitoring the scraping process. The solution requires adept handling of rate limits and the use of rotating proxies for efficient data scraping. Data To Be Scraped: - Scrape all specified job listing data points. - Store and manage the scraped data in a SQL database for effective access and organization.

I'm currently facing issues with my web crawler code. Specifically, it's not managing to scrape all the required data and it's throwing errors inconsistently. This code has been built to scrape information from certain e-commerce platforms. The ideal freelancer for this task should show: - Proficiency in Node.js - Extensive experience with Puppeteer and web crawling, especially focusing on e-commerce platforms - Exceptional debugging skills Your challenge will be to inspect the code, identify the underlying issues and then fix them to ensure a comprehensive and seamless data scraping process. I look forward to collaborating with someone who can effectively resolve the issues and optimize the performance of this web crawler.

your website is causing the load spikes in question. You can consider reviewing this on your side additionally. We have also noticed that there is a lot of bot/crawler access to the website. Access requests originating from bots are registered every 2 to 5 seconds.

Hi, Today i have created a Scrapy crawler to crawl the website , but i'm not able to crawl the website because when i run the crawler, any request i do to the website, i got the 403 error to me. I just neeed you fix my code to do a request at someway the website return the code 200 instead of 403. I'm sending my code attached. Send me a print screen in the chat showing you received code 200 using the same url, then i will hire you to send me your code.

Looking for a seasoned AWS expert to assist in optimizing EC2 instances, S3 storage, and Lambda functions for improved performance. The ideal candidate will have: Mainly working on Aws glue jobs , S3,lambda, Crawler, step functions, dynamo DB ,Clouldwatchs, Appflow, parameter store , API Gateways, ETL Jobs,splunk logs CloudFormation - important big bucket,Spunk, Jenkins pipelines* CI/CD * , YAML for creating templates and manifest files * Sql mandatory for date base * ⁠using Postman for API calls * ⁠Python and spark Musk know

...developer to create a specialized web crawler focused on extracting vital data from e-commerce websites, primarily targeting ''. Key Requirements: The web crawler must adeptly parse text-based content and efficiently navigate the site's structure to extract pertinent information and links. Extensive expertise in Web crawling, Web scraping, and Data mining is essential for this project's success. Prior successful engagements in web scraping projects centered on e-commerce platforms would be greatly advantageous. Proficiency in programming Python is required. Familiarity with web scraping frameworks and libraries like BeautifulSoup, Scrapy, or Selenium is highly desirable. While the initial focus is on '', the developed crawler sho...

I'm seeking an expert to help cr...an expert to help create a Dockerfile that runs Perl and DBD::mysql so that I can run a legacy website. This should be a simple task if you have the knowledge. Specific requirements include: - A Dockerfile is the only output required, roughly corresponding to mine - It should build and run in wsl, i.e. docker run -p 8080:80 image_name - It must use Apache, Perl and the CPAN module DBD::MySQL - It should have been tested on a MySql database, the file is provided to run on localhost:8080/ - A screenshot of the above is required to prove that it is working (you just need a MySql Table with fields Id and Name) Candidates should ideally have: - Extensive experience in Docker and a basic knowledge of Perl - Understanding of...

...tech support got the Import buttons on the forms fixed but suggests they may not be able to fix the XML & XML+XSL submission buttons at the bottom of the forms. Here's what they say about that: When the server is upgraded, the Perl modules in the new server might not be fully imported. That's why your website doesn't work because some Perl module dependencies might be deprecated. MochaHost support says: We highly suggest you talk about the error with your Web Developer to ensure they can help you add the missing path on the Perl module. Based on our check, your website form processing is missing pointing to the `File::Spec` module which has been installed on the server. It seems like that might be a fairly easy fix and I have asked...

...performance tuning, sizing and capacity planning, database backup and recovery in a Unix O/S using RMAN and Oracle export/import utilities, administration, and maintenance to provide backup and disaster recovery. • Knowledge of installation, configuration, and maintenance of multi-node Oracle RAC enabled databases, including Oracle ASM and Oracle CRS management. Knowledge of PL/SQL packages, shell, and Perl scripts for task automation, and proactive database monitoring and notification using Oracle OEM Grid Control. • Demonstrated written and oral communication skills reviewing and editing established database policies and procedures and developing new SOPs for database software installation and configuration. • Experience with Cloud(AWS/Azure/OCI) migrations (DB a...

I'm looking for seasoned coders that excel in Perl, to reconfigure my current Perl email script. The primary aim is to make it efficient in sending out emails using Google's SMTP. Key Deliverables: - Modify Perl script to use Google's SMTP via the Net::SMTP::SSL or Email::Send::SMTP::Gmail modules - Integrate secure connection to Gmail - Integrate Email authentication No requirement for handling attachments is necessary. Need to understand how the forms work. Proficiency in Perl, understanding of SMTP protocols, specifically Google's SMTP, and experience with Net::SMTP::SSL and Email::Send::SMTP::Gmail modules are crucial for success in this task.

...developer who can create an advanced website that's deeply interconnected with MS Azure. The website will need to have: 1. User registration and login: A system for users to create new accounts and log in as desired. 2. Chat functionality powered by GPT: A well-equipped Chat-bot that's capable of handling inquiries based on proprietary documents. 3. Comprehensive integration with LLM (GPT): A system that is able to process different types of documents (text, web-pages, PDFs etc.), inferring from both internal and external sources. 4. Web-link Web-Crawler: An engine that can gather data from the web and cross-link it with Azure. 5. Advanced SEO: Optimization techniques to help improve website ranking and visibility. 6. Google Maps integration: An op...

I'm seeking skilled Perl coders to modify my email script. This script currently sends out emails well after a web form submission using the Net::SMTP::SSL module. Success in this project will mean updating the script to instead utilize the Email::Send::SMTP::Gmail module. Ideal Skills: - Proficient in Perl coding - Familiarity with Net::SMTP::SSL and Email::Send::SMTP::Gmail modules - Understanding of Google SMTP Key Responsibilities: - Reconfigure Perl script to work with Email::Send::SMTP::Gmail module - Ensure the script remains efficient for sending out web form submission emails - Test script after modifications to verify seamless integration with Gmail SMTP

I'm looking for a Google Domains admin to help me fix an issue with page indexing and device permissions for my in-domain email addresses. Tasks: - Analyzing and resolving an issue with page indexing (the Google crawler is complaining about something but I can't tell what) - Setting device permissions for in-domain email addresses (I'm having trouble because users seem to be currently prohibited from logging in on their own devices and I can't find a setting to allow better functionality) Experience and Skills: - Expert knowledge of Google Domains - Proven track record of resolving similar issues Applications: - Only applications showcasing past work in Google domain administration will be considered. Please include the details of previous projects you have ...

This project involves conducting a dark web websites crawl with the primary objective of gathering dynamic intelligence and monitoring online activity. Key Details: - ACHE Crawler needs to be used to crawler dark web and collect available websites urls Required Skills and Experience: - You need to be proficient in Python programming and have previous experience in web crawling, especially on the dark web. - Familiarity with network security and understanding of dark web dynamics is highly preferred. - Familiarity with ACHE Crawler, Docker and Scrapy is needed. On completion, a detailed report of the findings, as well as the script code used, will be expected.

I am looking for an adept developer to create a data crawler that can efficiently harvest text data. The main area of focus for this project is to extract data from the MCA (Ministry of Corporate Affairs) of newly registered companies. Key Functionalities: • The crawler should be designed to extract text data. • It should specifically retrieve MCA data concerning newly registered companies. Expected Results: The solution should result in an efficient, accurate extraction of data from the designated sources, presented in a convenient, readily usable format. Timeline: The project needs to be completed within a month. Ideal Candidate: The ideal candidate for the project will have: • Extensive experience in data crawling and programming. • Demonstrated a...

As an indie game developer, I am in need of a Unity Game Designer to create multiple engaging, cave-like dungeons for a fantasy-themed dungeon crawler. The ideal candidate should possess: • Expertise in Unity3D and procedural level generation mechanisms • Creativity to design dungeons encapsulating a dark and eerie vibe cursed with an infused ancient mystery. Key Project Components: • Procedural Dungeon Creation: Dungeons must be designed using procedural generation techniques to bring forth the essence of unpredictable gameplay and replayability. Upon refreshing, a new layout should be generated. • Triple-trap system: The incorporation of multi-tier traps such as spikes and pitfalls, moving platforms, and hidden passages, is a must. The designer must come u...

I'm looking for a skilled developer to assist with a Google Earth project, focusing on user interface design. Main Tasks: - Designing the user interface Features R...project, focusing on user interface design. Main Tasks: - Designing the user interface Features Required: I would like a Google Maps web based application. Where the users are able to click features on a map using terrene view(for example trees) JavaScript to pick up the location and address of the property that’s been selected, and the address to be saved into a database using perl CGI in my SQL database. The perl component is optional. Areas that have previously been selected should be highlighted in a yellow highlight or similar The code should work on windows and a Unix platform. I will pro...

Hello, I'm seeking a professional with expertise in Perl who can integrate and manage the Gmail API for sending emails. Specifically, the tasks include: - Developing an application from scratch - Integrating the Gmail API to send emails - Activating basic email sending functionality with the Gmail API The ideal candidate would be proficient in Perl and have a solid understanding of APIs, particularly the Gmail API. Knowledge of email handling and management is desirable, but not necessary. This project requires the development of new software, so considerable experience in software development and project management will be beneficial. Please include details of your API integration experience in your bid.

I have a requirement to develop a Software as a Service (SaaS) solution which includes a crawler and a data presentation dashboard. Crawler: - The crawler needs to be designed to extract both text and images. - This data extraction has to happen on a daily basis. Dashboard: - The dashboard should present the data collected by the crawler in an understandable manner, ideally through visual graphs or charts. - It must also include a search function so specific data can be easily located. Ideal skills for this project include: - Proficiency in SaaS development. - Experience in buildling crawlers. - Strong knowledge in data visualization techniques. Please note, any additional recommendations for enhancements to this project are welcomed and appreciated....

...recommendations for the management of the focal firm. You are expected to use at least 250 reviews of each selected firm (not more than 1 year old review data). You may choose 3 different firms or 3 different units of the same firm. e.g., Taj hotel reviews across 3 different cities or reviews of 3 different hotels. For collecting recent review data, you are allowed to take external help (publicly available crawler scripts, scripts from your group members or batchmates). The actual data analysis scripts and report must be prepared based on your own original work. Prepare a report – in not more than 12 pages, excluding title, references, and appendix – outlining the steps you have followed while performing customer preference analysis. You can explore different text ana...

...years of Perl + XML + SQL experience, please submit your bid with exact monthly payment requirement. The budget range specifies our monthly budget per developer for full time ( 8 hours per day) commitment, Monday to Friday. Monthly budget is 80K to 120K per month depending on the experience. This job is open to developers from INDIA only. Bangalore based candidates who can work in hybrid mode will be given priority. The ideal candidate should have a minimum of 3 years of experience in Perl development and scripting, with expertise in XML and SQL (Preferred) Below is the Job Description for an in-depth understanding of the requirements. JOB DESCRIPTION: 3 – 5 years of proven knowledge of programming languages and operation systems as outlined Programming lang...

Check out my website for model download: What I will do with the wagon: My business goal is to create a software that can take any track of any roller coaster from the noLimit software, open it in my software, configure it, and click build. Then, I'll have a 3D printed track with all supports. I'll sell the software with the STL of the wagon. Futuristic Roller Coaster Wagon Design I'm looking for an experienced 3D designer specialized in Solidworks to create a body for a roller coaster wagon. This project requires a keen eye for aesthetics, as well as a solid understanding of mechanical engineering principles to incorporate a working seat restraint mechanism with a lock system. The design will be built upon an existing base that includes a differential-like

I'd like a web crawler that can go out to a list of US university websites and find documents (or pages) that describe certain aspects of university policy. I'd like the documents labeled and organized in a folder. I will provide a set of likely keywords and the domains to be searched. The domains will be provided in a spreadsheet containing university names and homepages. There are approximately 3000 domains to be searched. The keywords will be samples and may not be the exact names of the policy. A flexible search is required. A successful project will return the academic integrity or misconduct policy for each university as a PDF along with a link to the original document online. Note: this is part of a research project for a university professor.

I need an expert in Google Dork searches to assist me in finding a long-lost perl file. The file was from 2001. File name is: Some text within the file is: File Name: Version: 3.0b, Date: Friday May 18th 2001 *Look at source for more info. If you find it put in your bid "FOUND IT!" Pay is $40.00

I'm seeking a detailed-oriented freelancer to develop a data crawler capable of collecting specific information from various e-commerce websites and social media platforms. The objective is to harvest both text and numerical data efficiently. Key Requirements: - Target Data: Retrieve product prices as efficiently as possible. - Sources: Focus on well-known e-commerce platforms and social media sites where product listings are prevalent. - Essential Skills: Proficiency in web scraping tools, data extraction techniques, and handling bot detection mechanisms. - Experience: Past projects involving e-commerce or social media data scraping are highly beneficial. - Output Expectation: The data should be organized and provided in a structured format such as CSV, JSON, or Excel for fur...

...seeking a proficient Perl developer well-versed in web automation using Perl for a specific task. The project involves enabling automated liking of YouTube comments through Perl. Based on my discussion with ChatGPT, this task is feasible by providing the necessary parameters: 'YOUR_COMMENT_ID', 'YOUR_SESSION_ID', 'YOUR_XSRF_TOKEN', and 'YOUR_YOUTUBE_COOKIE'. If you possess the expertise to accomplish this, kindly reach out. To secure this project, a demonstration of your capabilities is required. The script is available in ChatGPT, and I can provide it upon request. While this initial task is relatively small, successful completion may lead to more extensive projects in the future. Please bid on this project only if you can ...

I am in need of a skilled freelancer to develop a code or software that continuously crawls for live stream data related to entertainment content. This program should be able to automatically retrieve, validate, and ensure the functionality of the live stream links across mobile platforms, specifically iOS and Android. **Key Requirements:** - Develop a crawler that targets live stream links dedicated to entertainment content. - Ensure the retrieved live stream links are live and functional 24/7. - The solution must be operational and fully compatible with iOS and Android platforms. **Ideal Skills and Experience:** - Proficiency in programming languages suitable for web crawling and automation (e.g., Python, JavaScript). - Experience in working with APIs and understanding of how to...

I need a skilled web crawler expert to extract specific information daily from a designated grocery website and deliver the collected data in a spreadsheet format. My primary focus is on the following details: - Product names - Prices - Images - Descriptions (including size and other relevant information) Ideal Skills and Experience: - Proficiency in web crawling tools and technologies - Experience in extracting and parsing large datasets - Ability to schedule and automate crawling tasks on a daily basis - Expertise in data formatting and spreadsheet management The extracted data should be neatly organized and presented in a spreadsheet, with clear and distinct columns for product names, prices, images, and descriptions. This job requires a high level of attention to detai...

?I cover the full spectrum of AWS Cloud consulting services, from assessments, strategy, and roadmap, to design, build, and migration – and from adoption, optimization, and management to data and analy...RDS, DynamoDB, Elasticache, Redshift, AWS Backup · Operating Systems: UNIX, Redhat LINUX, Windows · Networking & Protocols: TCP/IP, Telnet, HTTP, HTTPS, FTP, SNMP, LDAP, DNS, DHCP, ARP, SSL, IDM 6.0 and 7.0 · DevOps Tools: Puppet, Chef, Subversion (SVN), GIT, Jenkins, Hudson, Puppet, Ansible, Docker and Kubernetes · Scripting Languages: UNIX Shell Scripting (Bourne, Korn, C and Bash), Python, Perl Scripting, · Database: Oracle, Amazon Aurora, MySQL, MariaDB, PostgreSQL, Microsoft SQL Server · Version Controlling Tools: Subversion...

...seeking an experienced developer to create a web crawler specifically for the bookmaker site. The primary task for the crawler is to efficiently and accurately extract odds information focused on football events. Required Skills & Experience: - Proficiency in Python or another scripting language suitable for web crawling. - Previous experience working with or creating web crawlers. - In-depth understanding of the bookmaker betting industry, specifically for football. - Proven ability to extract data from complex websites effectively. Project Specifics: - The main data point to extract will be the betting odds. - The crawler should primarily focus on football events. - It is key that the crawler is designed to handle changes in the website struc...

Creating a fiverr crawler that scraps gigs, uses api to get gig details and using proxy exports data to the csv file using a desktop application..

We are looking for a Linux developer to join our team. We are all remote workers. We are looking for the following skills, most important at the top: Perl SQL (Mysql/MariaDB) Shell Script PHP / Codeigniter Linux Docker We operate a mixture of web sites (php/apache) and data manipulation scripts (mainly perl).

Poszukujemy osoby zdolnej do zaprogramowania narzędzia, które umożliwi nam analizę i selekcję informacji na stronach www. Celem jest stworzenie zaawansowanej bazy danych, która pozwoli na efektywne sortowanie i filtrację stron według specyficznych parametrów, takich jak poziom aktywności użytkowników. Naszym priorytetem jest zgromadzenie i organizacja informacji w sposób umożliwiający łatwe zarządzanie i dostęp do danych.

...shortcut, it would take me to my website. I got it working years ago and would edit it just to deal with shortcut additions, deletions & changes. I really don't know the first thing about programming. I copied my public_html folder from the old to new host. I made sure had 755 permission. The .httaccess came over also. The first line in .htaccess - #!/usr/local/bin/perl - I didn't / don't have a bin or perl folder anywhere in my directory tree on old OR new server My DNS zone has an a record for * pointing to my server's IP address, as before. When I type something like now, it redirects to: which is a cpanel page saying "**SORRY! If you are the owner of this website, please contact your hosting

We've a opening for a Scrapy Developer. Description: - Designing crawler for extracting data for various market sources. - Utilising country switcher whenever possible for multi version of sites. I,e en-us, zh-hk version of site. Desired skills: - Scrapy - Splash (nice to have) - Ajax/XHR API simulation for JS rendered site - Git/Bitbucket - Phyton - Jira (nice to have) First you make one crawler for a fixed price. You will receive access to a sample Bitbucket project with an example crawler. When the first crawler you've build is good we can continue on an ongoing basis (9 EUR per hour).

I'm seeking a Perl CGI expert to improve the security of my email script. The task requires integration of both SpamExperts and SMTP Authentication into the existing email script named wel.pl. Skills and Experience: - Familiarity with Perl CGI scripts - Experience in implementing SpamExperts and SMTP Authentication Tasks to be accomplished: - Configure script to use the SMTP details for sending emails - Integrate SpamExperts for real-time spam filtering - Implement SMTP Authentication for enhanced email security. My website host has integrated SpamExperts service with my Shared Hosting plan. It ensures that all emails that you send reach their destination and are delivered to the Inbox, not the Spam box of the person you send emails to. To use SpamExperts, t...

I am looking for an experienced developer who can design and implement an application that will break image-based captcha and in return issue a certificate. I need full professionalism on this since the system will be used to bring efficiency in our processes. The scope of work includes: - Designing a crawler that bypasses the image-based captcha - Test the crawler on different browsers to ensure efficiency - Documenting the testing processes and results - Identifying any potential risks and ways to mitigate them The ideal professional must: - Have demonstrable experience in designing and implementing crawlers - Have exceptional knowledge of image-based captcha - Show integrity, professionalism, and confidentiality in all the tasks - Be able to start work immediately - S...

...next level. Here's what I'm aiming for: **Required Customizations:** - Integration of advanced features that go beyond the basic user interface changes. This includes but is not limited to improving the platform's overall functionality and user experience. **Ideal Freelancer Qualities:** - Extensive experience with the XFileSharing script. - Strong background in web development, particularly in Perl, the language XFileSharing is written in. - Proficient in implementing secure file encryption methods to protect users' data. - Capability to implement multiple file upload features for a streamlined user experience. - Expertise in enabling social media integration, allowing users to share files directly on their social media channels. **Application Requirements...

We are looking for a crawler/scrapper for a specific android app. e.g. This app has a map which is overlaid by a marker on specific locations, once you click on it it shows further details like address, availability, amenities available etc. We would like a tool which on a given frequency (calibratable) pulls the data from the app and update in a excel file ( or similar database). The tool can be developed in any environment, but should be executable on Windows machine. The deliverable is the tool and its source code.

Generate a spreadsheet with ingredients, ingredient URLs, and total cost for 2,500 recipes as an initial POC to demonstrate that recipe data can be accessed from API provided by the client and then the crawler can retrieve and save the ingredient information from Tesco's website.

...a Python developer skilled in creating website crawlers/ scrapers. The objective of the project is to extract medical papers and information for competitive analysis and content monitoring purposes. Specific data points needed to be extracted are: - User comments/reviews - Postings or listings data Preferably, this information should be acquired from Google and any other online resources. The successful bidder should have experience in Python, web scraping, data extraction, and working with APIs. Knowledge in text mining and natural language processing will be an added advantage. Your scraper must be capable of handling hundreds of webpages effectively and efficiently. We have already completed 50% of the project as we have developed the crawler in-house using Python ...

Need of a proficient Python programmer to amend my application's data write execution output. Currently, the app writes output data in a CSV format and I want this changed to Excel. The App is weather related and data is extracted from grib files so experience in this would be beneficial. -Proficient in Python programming -Grib files knowledge beneficial -Strawberry perl knowledge beneficial -Change Write output files csv currently to excel Please submit your proposals along with how soon you can complete this project.

Description: We are in search of an experienced programmer with expertise in Python and JavaScript to develop a sophisticated script that operates akin to a "crawler." The primary function of this program will be to systematically scan specified Google search result links to identify LinkedIn profiles and extract pertinent contact information with email. Project Objective: The program will utilize Google search results to locate LinkedIn profiles and then proceed to retrieve contact data from these LinkedIn pages. It is imperative to note that such data collection practices are deemed legal, supported by a US court ruling which declares that information on LinkedIn is publicly accessible. Thus, there are no legal constraints; the data can be openly viewed and downloaded. ...

I need to scrap data from and other similar websites, all to be saved to ...includes name, description, location, size, sales, and at least 10 other fields and probably up to 40 fields (don't have a full inventory). This is a manual process, want to be able to update the catalog and the content manually on demand, can be on a terminal screen or in a visual enviroment. The script or system will be installed in my online Centos server, where you would be able to run Java, Perl, Python or Php. In any case I'd prefer PHP and then Python for me to be able to understand or auto support the script if an "emergency" appears. DB is MySQL 8 Now is only that site but there are several so I'll need to make this flexible enough to be able to add scrapping of new site...