Extracting data from Transfermarkt

Overview

This package is designed to allow users to extract various world football results and player statistics from the following popular football (soccer) data sites:

Installation

You can install the CRAN version of worldfootballR with:

install.packages("worldfootballR")

You can install the released version of worldfootballR from GitHub with:

# install.packages("devtools")
devtools::install_github("JaseZiv/worldfootballR")

library(worldfootballR)

Usage

Package vignettes have been built to help you get started with the package.

For functions to extract data from FBref, see here
For functions to extract data from Understat, see here
For functions to extract data for international matches from FBref, see here
For functions to load pre-scraped data, see here

This vignette will cover the functions to extract data from transfermarkt.com

Join FBref and Transfermarkt data

To be able to join data player between FBref and Transfermarkt, player_dictionary_mapping() has been created. There are over 6,100 players who have been listed for teams in the Big 5 Euro leagues on FBref since the start of the 2017-18 seasons, with all of these mapped together. This is expected to be updated and grow over time. The raw data is stored here

mapped_players <- player_dictionary_mapping()
dplyr::glimpse(mapped_players)

Transfermarkt Helper Functions

The following section will outline the various functions available to find different URLs to be able to pass through the Transfermarkt suite of functions outlined in this vignette.

Team URLs

To get a list of URLs for each team in a particular season from transfermarkt.com, the tm_league_team_urls() function can be used. If the country/countries aren’t available in the main data set, the function can also accept a League URL from transfermarkt.com. To get the league URL, use the filtering options towards the top of transfermarkt.com, select the country and league you want to collect data from, head to that page, and copy the URL.

team_urls <- tm_league_team_urls(country_name = "England", start_year = 2020)
# if it's not a league in the stored leagues data in worldfootballR_data repo:
league_one_teams <- tm_league_team_urls(start_year = 2020, league_url = "https://www.transfermarkt.com/league-one/startseite/wettbewerb/GB3")

Player URLs

To get a list of player URLs for a particular team in transfermarkt.com, the tm_team_player_urls() function can be used.

tm_team_player_urls(team_url = "https://www.transfermarkt.com/fc-burnley/startseite/verein/1132/saison_id/2020")

Staff URLs

To get a list of staff URLs for a particular team(s) and staff role in transfermarkt.com, the tm_league_staff_urls() function can be used.

The staff roles that can be passed to the function via the staff_role argument are below:

“Manager” (this will also return caretaker managers)
“Assistant Manager”
“Goalkeeping Coach”
“Fitness Coach”
“Conditioning Coach”

# get a list of team URLs for the EPL 2021/22 season
epl_teams <- tm_league_team_urls(country_name = "England", start_year = 2021)
# get all EPL managers for the 2021/22 season
epl_managers <- tm_team_staff_urls(team_urls = epl_teams, staff_role = "Manager")

# get all EPL goal keeping coaches for the 2021/22 season
epl_gk_coaches <- tm_team_staff_urls(team_urls = epl_teams, staff_role = "Goalkeeping Coach")

League Season-Level Data

This section will cover the functions to aid in the extraction of season team statistics and information for whole leagues.

League Table by Matchdays

To be able to extract league tables for select matchday(s), the below function can be used.

The function can accept either the country name, season start year and matchday number(s), or for leagues not contained in the worldfootballR_data repository, it can accept the league URL, season start year and matchday number(s).

#----- to get the EPL table after matchday 1 of the 20/21 season: -----#
epl_matchday_1_table <- tm_matchday_table(country_name="England", start_year="2020", matchday=1)
dplyr::glimpse(epl_matchday_1_table)

# #----- to get the EPL table after each matchdays from matchday 1 to matchday 35 of the 20/21 season: -----#
# epl_matchday_1to35_table <- tm_matchday_table(country_name="England", start_year="2020", matchday=c(1:35))

#----- to get the League One table after each matchdays from matchday 1 to matchday 5 of the 20/21 season: -----#
league_one_matchday_1_table <- tm_matchday_table(start_year="2020", matchday=1:5,
                                                 league_url="https://www.transfermarkt.com/league-one/startseite/wettbewerb/GB3")
dplyr::glimpse(league_one_matchday_1_table)

League Debutants

To be able to extract all debutants making either their league or professional debut, the tm_league_debutants() function can be used.

To see all league debutants (regardless of their professional status), set debut_type = "league", while setting debut_type = "pro" will only return debutants in the selected league who are making their professional debuts.

The variables debut_start_year and debut_end_year set time periods on when debutant data is required for. As with all transfermarkt functions, this season value is the starting year if the season, ie for the 2021-22 LaLiga season, this value is set to 2021.

# Laliga players making their LaLiga debut in 2021/2022
laliga_debutants <- tm_league_debutants(country_name = "Spain", debut_type = "league", debut_start_year = 2021, debut_end_year = 2021)
dplyr::glimpse(laliga_debutants)

# English League One players making their PRO debuts in 2021/2022
league_one_PRO_debutants <- tm_league_debutants(country_name = "", league_url = "https://www.transfermarkt.com/league-one/startseite/wettbewerb/GB3", debut_type = "pro", debut_start_year = 2021, debut_end_year = 2021)
dplyr::glimpse(league_one_PRO_debutants)

Expiring Contracts

To be able to extract a list of all players whose contracts expire in a selected year, the tm_expiring_contracts() can be used.

Set the contract_end_year to be equal to the calendar year the contracts are due to expire.

#----- LaLiga players with expiring contracts in 2022: -----#
laliga_expiring <- tm_expiring_contracts(country_name = "Spain", contract_end_year = 2023)
dplyr::glimpse(laliga_expiring)

#----- Can even do it for non-standard leagues - English League One players with expiring contracts in 2022: -----#
# league_one_expiring <- tm_expiring_contracts(country_name = "",
#                                                contract_end_year = 2023,
#                                                league_url = "https://www.transfermarkt.com/league-one/startseite/wettbewerb/GB3")

League Injuries

To get a list of all reported current injuries for a selected league, use the tm_league_injuries() function:

# to get all current injuries for LaLiga
laliga_injuries <- tm_league_injuries(country_name = "Spain")
dplyr::glimpse(laliga_injuries)

#----- Can even do it for non-standard leagues - get all current injuries for League One in England
# league_one_injuries <- tm_league_injuries(country_name = "",
#                                                league_url = "https://www.transfermarkt.com/league-one/startseite/wettbewerb/GB3")

Team Data

This section will cover off the functions to get team-level data from Transfermarkt.

Transfer activity by team

To get all the arrivals and departures for a team (or teams) in a season and data regarding the transfer (transfer value, contract length, where they came from/went to, etc), the tm_team_transfers() function can be used. This function can return either summer, winter or all for both transfer_windows:

#----- for one team: -----#
bayern <- tm_team_transfers(team_url = "https://www.transfermarkt.com/fc-bayern-munchen/startseite/verein/27/saison_id/2020", transfer_window = "all")
dplyr::glimpse(bayern)

#----- or for multiple teams: -----#
# team_urls <- tm_league_team_urls(country_name = "England", start_year = 2020)
# epl_xfers_2020 <- tm_team_transfers(team_url = team_urls, transfer_window = "all")

Squad Player Stats

To get basic statistics (goals, appearances, minutes played, etc) for all games played by players for a squad season, the tm_squad_stats() function can be used:

#----- for one team: -----#
bayern <- tm_squad_stats(team_url = "https://www.transfermarkt.com/fc-bayern-munchen/startseite/verein/27/saison_id/2020")
dplyr::glimpse(bayern)

#----- or for multiple teams: -----#
# team_urls <- tm_league_team_urls(country_name = "England", start_year = 2020)
# epl_team_players_2020 <- tm_squad_stats(team_url = team_urls)

Player Valuations

To get player valuations for all teams in a league season, use the tm_player_market_values() function:

#----- Can do it for a single league: -----#
a_league_valuations <- tm_player_market_values(country_name = "Australia",
                                       start_year = 2021)
dplyr::glimpse(a_league_valuations)

#----- Can also do it for multiple leagues: -----#
# big_5_valuations <- tm_player_market_values(country_name = c("England", "Spain", "France", "Italy", "Germany"),
#                                        start_year = 2021)

#----- Can also do it for non standard leagues: -----#
# league_one_valuations <- tm_player_market_values(country_name = "",
#                                        start_year = 2021,
#                                        league_url = "https://www.transfermarkt.com/league-one/startseite/wettbewerb/GB3")

Player Data

This section will cover the functions available to aid in the extraction of player data.

Player Bios

To get information about a player, like their age, foot, where they were born, who they play for, their contract details, social media accounts and a whole lot more, use the tm_player_bio() function.:

#----- for a single player: -----#
hazard_bio <- tm_player_bio(player_url = "https://www.transfermarkt.com/eden-hazard/profil/spieler/50202")
dplyr::glimpse(hazard_bio)

#----- for multiple players: -----#
# # can make use of a tm helper function:
# burnley_player_urls <- tm_team_player_urls(team_url = "https://www.transfermarkt.com/fc-burnley/startseite/verein/1132/saison_id/2020")
# # then pass all those URLs to the tm_player_bio
# burnley_bios <- tm_player_bio(player_urls = burnley_player_urls)

Player Injury History

To be able to get an individual player(s) injury history from transfermarkt, use the tm_player_injury_history() function.

#----- for a single player: -----#
hazard_injuries <- tm_player_injury_history(player_urls = "https://www.transfermarkt.com/eden-hazard/profil/spieler/50202")
dplyr::glimpse(hazard_injuries)

#----- for multiple players: -----#
# # can make use of a tm helper function:
# burnley_player_urls <- tm_team_player_urls(team_url = "https://www.transfermarkt.com/fc-burnley/startseite/verein/1132/saison_id/2021")
# # then pass all those URLs to the tm_player_injury_history
# burnley_player_injuries <- tm_player_injury_history(player_urls = burnley_player_urls)

Player Transfer History

To be able to get an individual player(s) transfer history from transfermarkt, use the tm_player_transfer_history() function.

The parameter get_extra_info allows users to decide if they want to scrape the extra info regarding the transfer. This will give additional information regarding contract length and nations involved, but will require more time. The default value is TRUE.

#----- for a single player, get_extra_info defaulting TRUE: -----#
jack_rodwell_transfer_history <- tm_player_transfer_history(player_urls = "https://www.transfermarkt.com/jack-rodwell/profil/spieler/57079")
dplyr::glimpse(jack_rodwell_transfer_history)

#----- for a single player, get_extra_info FALSE: -----#
jack_rodwell_transfer_history <- tm_player_transfer_history(player_urls = "https://www.transfermarkt.com/jack-rodwell/profil/spieler/57079", get_extra_info = FALSE)
dplyr::glimpse(jack_rodwell_transfer_history)

#----- for multiple players, get_extra_info FALSE: -----#
all_leeds_united_links <- tm_team_player_urls(team_url = "https://www.transfermarkt.com/leeds-united/startseite/verein/399")
all_leeds_united_players_transfer_history <- tm_player_transfer_history(all_leeds_united_links, get_extra_info = FALSE)
dplyr::glimpse(all_leeds_united_players_transfer_history)

Player Absence

To be able to get a player’s (or players’) absence history as a result of suspensions from transfermarkt, use the tm_get_player_absence() function.

#----- for a single player: -----#
romero_absence <- tm_get_player_absence(player_urls = "https://www.transfermarkt.com/cristian-romero/profil/spieler/355915")
dplyr::glimpse(romero_absence)

#----- for multiple players: -----#
# # can make use of a tm helper function:
spurs_player_urls <- tm_team_player_urls(team_url = "https://www.transfermarkt.com/tottenham-hotspur/startseite/verein/148")
# # then pass all those URLs to the tm_get_player_absence
spurs_player_absence <- tm_get_player_absence(player_urls = spurs_player_urls)

Club Staff Data

From version 0.4.7, users now have the ability to get historical data for club staff from transfermarkt.

The following two functions can be used, depending on the need (in addition to the helper function tm_team_staff_urls() detailed above).

Club Staff History

You can extract all employees by role in a club’s history using tm_team_staff_history().

The list of roles that can be passed to the staff_roles argument can be found here, and they’re also listed below:

# get a list of team URLs for the EPL 2021/22 season
epl_teams <- tm_league_team_urls(country_name = "England", start_year = 2021)
#----- then use the URLs to pass to the function, and select the role you wish to see results for: -----#
club_manager_history <- tm_team_staff_history(team_urls = epl_teams, staff_role = "Manager")
dplyr::glimpse(club_manager_history)

#----- can also get other roles: -----#
# club_caretaker_manager_history <- tm_team_staff_history(team_urls = epl_teams, staff_role = "Caretaker Manager")

Staff Member’s History

To be able to get all roles held by a selected staff member(s), the tm_staff_job_history() function can be used.

The function accepts one argument, staff_urls, which can be extracted using tm_team_staff_urls() explained in the helpers section above.

# get a list of team URLs for the EPL 2021/22 season
# epl_teams <- tm_league_team_urls(country_name = "England", start_year = 2021)

# get all EPL goal keeping coaches for the 2021/22 season
epl_gk_coaches <- tm_team_staff_urls(team_urls = epl_teams[1:3], staff_role = "Goalkeeping Coach")

# then you can pass these URLs to the function and get job histories for the selected staff members
epl_gk_coach_job_histories <- tm_staff_job_history(staff_urls = epl_gk_coaches)
dplyr::glimpse(epl_gk_coach_job_histories)

Jason Zivkovic

2024-01-18