_private/qwestly-docs/Engineering/LinkedIn-Profile-Scraper.md

Table of Contents

LinkedIn Profile Scraper

The LinkedIn Profile Scraper is a Chrome extension that extracts structured profile data from LinkedIn profiles and saves it directly to Qwestly's candidate database.

Overview

The extension allows you to quickly capture comprehensive profile information from any LinkedIn profile page with a single click. It extracts data including work experience, education, skills, location, and more, then displays it in a review modal before saving to Qwestly.

Installation

  1. Load the extension in Chrome:

    • Open chrome://extensions
    • Enable "Developer mode" (toggle in the top right)
    • Click "Load unpacked"
    • Select the project directory

  2. The extension icon will appear in your Chrome toolbar

Usage

Scraping a Profile

  1. Navigate to any LinkedIn profile page (URLs that start with https://www.linkedin.com/in/...)
  2. Click the extension icon in your Chrome toolbar
  3. A modal will appear displaying the extracted profile data in JSON format
  4. Review the data to ensure accuracy
  5. Click "Save to Qwestly" to save the profile to the candidates database

What Gets Extracted

The extension captures comprehensive profile information including:

Personal Information:

  • Full name, first name, last name
  • Headline
  • Location (city, state, country)
  • Profile image URL
  • About/bio section
  • Connection count
  • Profile verification status
  • Premium/Creator/Influencer badges

Current Position:

  • Job title
  • Company name
  • Company LinkedIn URL
  • Company logo
  • Start date (month and year)
  • Job duration

Work Experience:

  • Complete work history with:
    • Job titles
    • Company names
    • Employment dates
    • Job descriptions
    • Company LinkedIn URLs
    • Company logos

Education:

  • Schools attended
  • Degrees earned
  • Fields of study
  • Graduation dates
  • School LinkedIn URLs
  • School logos
  • Activities and societies

Additional Data:

  • Languages
  • LinkedIn profile URL
  • Profile ID and public identifier

Saving Profiles

When you click "Save to Qwestly":

  1. The button will show a loading state
  2. The profile data is sent to Qwestly's API (https://discover.qwestly.com/api/linkedin/save-profile)
  3. On success, the button turns green and shows "Saved Successfully!"
  4. The modal can be closed by clicking the × button or clicking outside the modal

Limitations

  • LinkedIn Updates: LinkedIn frequently updates their page structure. If the extension stops working, it may need updates to match LinkedIn's new HTML structure or data format.

  • Hidden Content: The extension only captures content that is visible on the page. If sections are hidden behind "Show more" buttons or require scrolling to load, they may not be captured.

  • Page Load: Ensure the profile page has fully loaded before clicking the extension icon for best results.

Troubleshooting

Extension doesn't work:

  • Make sure you're on a LinkedIn profile page (linkedin.com/in/...)
  • Refresh the page and try again
  • Check the browser console for error messages

Data appears incomplete:

  • Scroll through the entire profile page to ensure all sections are loaded
  • Click any "Show more" buttons to expand hidden sections
  • Refresh the page and try again

Save button doesn't work:

  • Check your internet connection
  • Verify you have access to the Qwestly API
  • Check the browser console for error messages

Support

For technical issues or questions about the extension's functionality, refer to the project's GitHub repository for developer documentation and issue reporting.
https://github.com/Qwestly/linkedin-profile-scraper