Frequently Asked Questions
General Questions
In this section, we've collected answers to general questions about job posting data and Techmap. Learn more about the importance of job postings, where our data comes from, and the scope of our offerings.
About Job Postings
- Why do I need job postings?
- Jobs postings do provide valuable insights into the hiring activities, recruitment strategy, and technology usage of businesses as well as the job market in general. They can also be used to analyze your competition or monitor industry trends.
- Where does the data come from?
- Our company obtains data by crawling the internet and gathering job postings from various public sources, such as company websites, job boards, social media platforms, and career pages. This process involves identifying specific information on each job posting page, which may have a different layout and structure from other pages. We then convert this information into a standardized format, enhance it as needed, and store it in a common data structure for easy access and analysis.
- How far back does your data go?
- We have job postings from 2020 onwards in our database, which we built by gradually adding data sources over time. We update it daily by crawling all sources for the latest job postings. To learn more about the historical data, please explore the data for the countries you're interested in or email our Data Team to learn more about historical data.
- What is job posting data?
- Job postings, also known as job ads or job listings, are advertisements created by employers or recruiters to attract potential candidates for open job positions within their organizations. Job posting data is the information extracted from these job postings, gathered from various sources, and presented in a standardized format. This data typically includes details such as job title, location, job description, qualifications, company benefits, and educational requirements.
About Techmap
- What do you offer?
- Techmap offers customers access to job posting data that we collect for our tech workplace search engine in the form of data feeds, historical datasets, and an API. Our customers, primarily data scientists, can use this data for various use cases to gain insights, trends, and patterns about their competitors, potential leads, or the job market.
- What types of job postings do you offer?
- We offer a wide range of job postings across various industries and countries. Our database includes job postings for full-time, part-time, remote, and contract positions, as well as internships and entry-level roles. We also have job postings from both established companies and startups.
- Which countries do you cover?
- Over the years we've collected data from 198 countries, however, we only rarely catch jobs for some small countries. Currently we have roughly 92 countries with more than 100 jobs per month and 66 with more than 1000 jobs per month. More up-to-date information can be found in our data explorer.
Technical Questions
Here we cover technical questions related to Datafeeds, Datasets, and the Data API, as well as questions about data updates, quality, and usage.
About Datafeeds & Datasets
- What is a datafeed?
- Our datafeed consists of exported data files that we create on a daily basis for each country. Data files are compressed JSON files with one job posting per line. Subscribers can download these files and use the contained job postings as they see fit.
- What is a dataset?
- Our datasets consist of daily exports of job posting data files, which are available for each country we cover. The data files are compressed JSON files with one job posting per line, and customers can download these files and use the job postings as they wish.
- What data formats do you support?
- Currently, we only support JSON as a data format in our data files and via our API, which is a wide-spread standard to integrate our job posting data into your workflows and systems. If you are interested in differnt formats such as CSV, YAML, XML, Avro, Parquet, etc. please contact our Data Team.
- Does the Datafeed also provide historical data?
- When you subscribe to our data feed, you will have access to all of our data files from the month you make payment onwards. Our data files are available for a minimum of one year from your subscription start date. You will be able to access these files at any time during this period using the credentials we provide you with.
- How many rows of data will I receive?
- We do not limit the number of job postings you receive from us. You get all the job postings we find for the countries you paid for. However, the number of job postings varies by country. Currently, you would receive approximately 2 million job postings per month for the USA, but only 122 thousand for Ireland.
About the Data API
- What can I do with the API?
- Techmap's API allows customers to programmatically access our job posting data. This means that you can use our API to integrate our data into your own software or web applications. With the API, you can query our database for specific job postings or extract data for particular countries, companies, or industries. You can use this data to build your own job search engine, analyze job market trends, or create custom reports for your business needs. The API also enables you to receive real-time updates of new job postings, so you can stay on top of the latest job market developments.
- How is the API billed and metered?
- The API is billed based on usage and is metered by the number of API requests made. Each API request counts towards your usage, and you will be billed based on the number of requests you make during a billing cycle. The cost per request can vary depending on your subscription plan and usage volume. For more information please see our pricing page for the API on RapidAPI.
- What data formats does the API support?
- The API supports the JSON format for both request and response payloads. This means that when you make a request to the API, you need to pass your query parameters in JSON format, and when you receive a response, it will be in JSON format as well. The advantage of using JSON is that it is a lightweight and widely supported format that is easy to parse and manipulate using a variety of programming languages. This allows you to integrate the API seamlessly into your existing workflows and applications without having to worry about compatibility issues.
- Does the API also provide historical job data?
- Yes, the Job API provides historical job data in addition to the real-time data. The historical data includes all the job postings that have been collected and processed since the start of 2020. This means that customers can access a vast amount of job posting data from the past, and use it to analyze job market trends, identify patterns, and gain insights.
- How many rows can I access with the API?
- We do not limit the number of job postings you can access, however, the API is metered and you have to pay for every accessed job posting. You can access all the job postings we've found for all countries since we began collecting them.
About Data Updates & Quality
- How often do you update your data?
- We update our data on a daily basis, so you can be sure that you're getting the most current job posting information available. Our data is collected from various sources, including company career pages, job boards, and social media, and we use advanced algorithms to ensure the accuracy and relevance of the data we provide. The refresh rates depend on scraping speed we can achieve with available resources.
- What delivery frequency options do you offer?
- We provide daily delivery frequency for our datafeeds and real-time frequency for our API. Data files are exported two days after the day finishes on UTC+2 to insure all jobs from data pipelines etc. have been collected. With datasets, customers have the option to access data from the whole month as daily data files.
- What data do you collect from job postings?
- The job posting data includes essential data fields such as source, country code, job title, URL, job posting text and HTML, original JSON objects, job position details (name, contract type, work type, career level), salary details (amount and period), contact information, tags, job location details (address, city, state, country, zip code), company details (name, description, website), creation date, and scraping date. Some of these fields are available via Datafeeds and Datasets, while others are only accessible through the API (see our Data Overview).
About Data Usage?
- How do I access the data files?
- Techmap provides data files that are stored on AWS S3 and are accessible via credentials we provide to customers after purchase. Customers can access these files through the terminal or shell scripts (See our developer section), which typically requires basic technical knowledge.
- How do I access and use the JSON data?
- Once you have downloaded the data files, you can process them using a program or shell script, decompress the files, parse the JSON data, and use it for various purposes such as storing them in your database or converting them into CSV files (for more information, check out our See our developer section). This task usually requires some basic technical knowledge.
- How can I access the API?
- To use the Techmap API on RapidAPI, follow these steps:
- Go to RapidAPI.com and sign up for an account if you haven't already.
- Go to our API page and subscribe to the API plan that best fits your needs.
- Copy the API key provided by RapidAPI.
- Go to the Techmap API documentation on RapidAPI and follow the instructions for making API requests using your API key.
Legal and Business Questions
This section addresses some common questions about legal and business issues, and covers topics related to our contracts, pricing, billing, trials, and samples. As a disclaimer, please note that we are not legal professionals and laws vary by country and state.
About Contracts
- How can I subscribe to the Job Datafeed?
- To subscribe to a data feed you can contact our Data Team and share your requirements. Our team will promptly get in touch with you, provide you with relevant data samples if required, and send you a contract. Let us help you get started on your data-driven journey.
- How can I buy historical Job Datasets?
- To buy datasets for specific countries and date ranges please contact our Data Team and share your requirements. Our team will promptly get in touch with you, provide you with relevant data samples if required, and send you a contract. We're eager to help you start utilizing data to make informed decisions.
- How can I access the Job API?
- To access our API for job posting data, please visit our API page and subscribe to the API plan that best fits your needs. Alternatively you can contact our Data Team and share your requirements to identify the perfect plan for you. Let us help you leverage the power of job posting data for your business.
- What is your termination policy?
- Typically, we offer a one-month termination period for the API and datafeeds - except for special discounts. We do not offer refunds for any unused portion of the current billing period. However, you will continue to have access to the datafeed until the end of the billing period.
About Billing, Pricing and Payments
- What are your pricing plans?
- Our pricing plans for job data include a subscription fee of €300 per month per country for job datasets and data feeds, and an API-based access to the database starting at 0.002€ per job record, with all countries included and no hard limits. We also offer a variety of RapidAPI plans that might match your specific application needs.
- What payment methods do you accept?
- Currently, we only support bank transfers. If you are interested in differnt payment methods such as Credit Cards, PayPal, etc. please contact our Data Team.
- Can I receive an invoice for my purchase?
- Yes! We provide invoices for all transactions, whether it is a dataset purchase or a subscription to our datafeed. You will receive an invoice for your dataset purchase via email, and for datafeeds, an invoice will be sent to you every month at the beginning of the month. For API-related invoices, RapidAPI will handle them. If you have any specific requirements for the invoice, please inform us, and we will do our best to fulfill your needs.
- Do you offer any discounts?
- We only offer discounts on bulk purchases and long-term contracts. Please contact our Data Team to discuss your requirements and see how we can provide the best value for your needs.
About Trials and Samples
- Do you offer trials for datafeeds?
- We do not offer trials at the moment. However, we can provide you with relevant data samples upon request to help you assess the quality of our job posting data. Additionally, you may also download some of our datasets from Kaggle to further explore and evaluate our offerings. If you have any questions or concerns, please feel free to contact our Data Team at any time.
- Do you offer a free API trial?
- You can try out our APIs for free on RapidAPI. We offer 100 free API calls, and you can see the pricing details on our RapidAPI pricing page. It's a great way to get started and see how our APIs can help you.
- Do you offer data excerpts?
- Sure, we offer data samples or excerpts upon request for some of our datasets. You can also download some of our datasets from Kaggle to further explore and evaluate our offerings. However, if you need a more specific sample, please don't hesitate to contact our Data Team and let us know which dataset you are interested in. Our team will get back to you with a relevant sample to help you make an informed decision about purchasing the full dataset.
About Data Privacy & Legitimacy
- Is the job posting data GDPR conform?
- Yes, job postings have a business context and contain no information on private individuals located in the European Union (EU). However, if you use the job postings to backfill your Job Board etc. and receive information from private individuals you must comply with GDPR.
- Is it legal to use scraped job postings?
- Yes, using scraped job postings is legal as web crawling and scraping from publicly available sources is allowed in the EU. We take measures to ensure that our data is collected in compliance with applicable laws and regulations, including GDPR. If you have any specific concerns about legal compliance, we recommend consulting with a legal professional.