Job Data Overview
Our Data
Our data can be accessed as a stream of daily data files for each country. You can subscribe to Datafeeds of individual countries, buy historical data starting from 2020 as Datasets, or access all data via our API! Furthermore, since July 2023 offer our data on AWS Data Exchange (ADX).
Data Samples / Excerpts
To get first impression and test the workflow you can access our Luxembourg dataset for free. Luxembourg largely dependent on the banking, steel, and industrial sectors. As of 2023, Luxembourg's population stands at 650k with a GDP per capita of 132k USD. The dataset starts January 1st, 2020, comprises of approx. 250k job ads, and grows by roughly 10k job listings monthly.
Alternatively, we have published several Datasets on Kaggle that you can explore:
- US Job Postings from May 5th 2023 (33k records; 805 MB GZip file - 4.46 GB uncompressed JSON)
- Ireland Job Postings from October 2022 (37k records; 101 MB GZip file)
- Ireland Job Postings from October 2021 (25k records; 56 MB GZip file)
- Ireland Job Postings from October 2020 (30k records; 58 MB GZip file)
- International Job Postings from September 2021 (3.4m records; 8 GB GZip file - 50 GB uncompressed)
or download a small JSON file with upto 3 records from 30 sources and 144 countries within April 2023:
- Sample of Job Postings from April 2023 (1.4k records; 3.3 MB JSON file)
If you need a more specific sample, please don't hesitate to contact us at data@techmap.io and let us know which dataset you are interested in. Our team will get back to you with a relevant sample to help you make an informed decision about purchasing the full dataset.
Data Fields
We store job postings as JSON in our Database and export it in daily data files with a more compact subset of essential data fields. We make them available via our API and we export them in daily datafiles with a subset of essential data fields. With our job posting data, you'll have all the information you need to build cutting-edge applications and drive success for your company.
Exported Data Fields (via Datafeeds & Datasets)
Our job postings in the datafiles is stored using JSON Lines, a popular format for structured data. Each posting contains a comprehensive set of essential data fields that can be used to extract valuable insights and inform your business decisions.
| Field Key | Field Type | Percentage | Description |
|---|---|---|---|
| source | String | 100.00% | The origin of the job posting with a countrycode such as 'monster_us'. |
| sourceCC | String (ISO3166) | 100.00% | The countrycode (ISO3166) the job posting is located, e.g., 'us'. |
| idInSource | String | 100.00% | The ID used in the source the job posting originated from. |
| name | String | 100.00% | The title of the job posting. |
| url | String (URL) | 100.00% | The link where we found the job posting. |
| text | String | 100.00% | The text of the job posting extracted from the html field. |
| html | String (HTML) | 100.00% | The original HTML used on the Website or stored in the JSON of the job posting page. |
| json | Object | 100.00% | The original JSON objects found in the job posting page including schema.org job posting data if available. |
| referenceID | String | 30.40% | An ID stated on the job postings page given by the original company for internal use. |
| position.* | Object | 99.96% | Data concerning the job position such as name, contract type (e.g., Permanent), work type (e.g., full-time), or career level (e.g., Junior). |
| salary.* | Object | 22.91% | Data concerning the job salary such as amount and period (e.g., 'Weekly'). |
| contact.* | Object | 30.83% | Data about contact information such as contact name, email, phone, or physical address. |
| orgTags.* | Object | 99.73% | Tags found on the job posting page (e.g., skills such as 'Java'), in the JSON (e.g., benefits) or from the browsing hierarchy (e.g., 'IT Jobs'). |
| location.orgAddress.* | Object | 100.00% | Data about the location of the job as originally stated on the job posting or its JSON. |
| company.* | Object | 100.00% | Data about the company of the job as originally stated on the job posting or its JSON. |
| locale | String | 28.24% | The language locale the job posting is written for (might not always be correct). |
| dateCreated | Date | 100.00% | The date and time the job posting was created by the company (as stated on the page or its JSON). |
| dateScraped | Date | 100.00% | The date and time the job posting was loaded and analyzed. |
Database Data Fields (via API)
The job postings in our database are stored using JSON and contain the following data fields (generated using variety.js from an April 2023 data slice over all countries). Please note that we do not list some internal data fields as well as the JSON taken from the job posting page.
| Field Key | Field Type | Counted | Percentage |
|---|---|---|---|
| _id | ObjectId | 8 M | 100.00% |
| contact | Object | 3 M | 30.83% |
| contact.address | String | 74 K | 0.89% |
| contact.email | String | 845 K | 10.06% |
| contact.name | String | 281 K | 3.34% |
| contact.phone | String | 2 M | 24.12% |
| dateCreated | Date | 8 M | 100.00% |
| dateExpired | Date | 4 M | 48.71% |
| dateScraped | Date | 8 M | 100.00% |
| datesUpdated | Array | 452 K | 5.38% |
| html | String | 8 M | 100.00% |
| idInSource | String | 8 M | 100.00% |
| json | Object | 6 M | 73.42% |
| lastUpdated | Date | 61 K | 0.73% |
| locale | String | 2 M | 28.24% |
| name | String | 8 M | 100.00% |
| orgAddress | Object | 8 M | 100.00% |
| orgAddress.addressLine | String | 8 M | 100.00% |
| orgAddress.city | String | 6 M | 75.15% |
| orgAddress.companyName | String | 8 M | 100.00% |
| orgAddress.country | String | 8 M | 98.31% |
| orgAddress.countryCode | String | 8 M | 95.65% |
| orgAddress.county | String | 8 M | 99.55% |
| orgAddress.district | String | 8 M | 98.04% |
| orgAddress.formatted | String | 8 M | 100.00% |
| orgAddress.geoPoint | Object | 2 M | 28.98% |
| orgAddress.geoPoint.lat | Number | 2 M | 28.98% |
| orgAddress.geoPoint.lng | Number | 2 M | 28.98% |
| orgAddress.houseNumber | String | 8 M | 99.49% |
| orgAddress.level | Number | 8 M | 100.00% |
| orgAddress.postCode | String | 3 M | 37.29% |
| orgAddress.quarter | String | 8 M | 99.09% |
| orgAddress.source | String | 933 K | 11.11% |
| orgAddress.state | String | 6 M | 66.03% |
| orgAddress.street | String | 4 M | 41.80% |
| orgCompany | Object | 8 M | 100.00% |
| orgCompany.description | String | 6 M | 73.49% |
| orgCompany.idInSource | String | 8 M | 100.00% |
| orgCompany.imgCover | String | 8 M | 98.90% |
| orgCompany.imgLogo | String | 8 M | 98.97% |
| orgCompany.info | Object | 809 K | 9.63% |
| orgCompany.info.careerpageURL | String | 16 K | 0.19% |
| orgCompany.info.companyEmailJobs | String | 101 K | 1.20% |
| orgCompany.info.companyEmailMain | String | 101 K | 1.20% |
| orgCompany.info.companyPhoneMain | String | 101 K | 1.20% |
| orgCompany.info.companySize | String, Number | 707 K | 8.42% |
| orgCompany.info.companySizeRange | String | 166 K | 1.98% |
| orgCompany.info.companyType | String | 306 K | 3.64% |
| orgCompany.info.dateIncorporated | Date | 154 K | 1.84% |
| orgCompany.info.foundingDate | Number | 154 K | 1.84% |
| orgCompany.info.revenue | String | 296 K | 3.52% |
| orgCompany.name | String | 8 M | 100.00% |
| orgCompany.nameOrg | String | 8 M | 100.00% |
| orgCompany.orgTags | Object | 306 K | 3.64% |
| orgCompany.orgTags.INDUSTRIES | Array | 306 K | 3.64% |
| orgCompany.registryID | String | 8 M | 98.45% |
| orgCompany.source | String | 8 M | 100.00% |
| orgCompany.sourceCC | String | 8 M | 100.00% |
| orgCompany.url | String | 4 M | 42.33% |
| orgTags | Object | 8 M | 99.73% |
| orgTags.CATEGORIES | Array | 5 M | 64.52% |
| orgTags.COMPANY_BENEFITS | Array | 273 K | 3.26% |
| orgTags.COMPANY_TYPES | Array | 6 K | 0.07% |
| orgTags.CONTRACT_TYPES | Array | 86 K | 1.03% |
| orgTags.INDUSTRIES | Array | 3 M | 36.63% |
| orgTags.JOBNAMES | Array | 3 M | 35.34% |
| orgTags.KEYWORDS | Array | 2 M | 21.70% |
| orgTags.LANGUAGES | Array | 50 K | 0.59% |
| orgTags.QUALIFICATIONS | Array | 644 K | 7.67% |
| orgTags.REQUIREMENTS | Array | 462 K | 5.51% |
| orgTags.RESPONSIBILITIES | Array | 77 K | 0.91% |
| orgTags.SKILLS | Array | 679 K | 8.09% |
| orgTags.WORK_TYPES | Array | 75 K | 0.90% |
| position | Object | 8 M | 99.96% |
| position.careerLevel | String | 2 M | 23.09% |
| position.contractType | String | 2 M | 26.72% |
| position.department | String | 1 M | 11.93% |
| position.name | String | 8 M | 99.79% |
| position.workType | String | 5 M | 60.73% |
| referenceID | String | 3 M | 30.40% |
| salary | Object | 2 M | 22.91% |
| salary.period | String | 1 M | 12.08% |
| salary.text | String | 1 M | 15.17% |
| salary.value | Number | 1 M | 12.01% |
| source | String | 8 M | 100.00% |
| sourceCC | String | 8 M | 100.00% |
| text | String | 8 M | 100.00% |
| url | String | 8 M | 100.00% |