Advancing UN Comtrade for physical trade flow analysis: Addressing the issue of missing values (2022)


RegisterSign in


  • Access throughyour institution

Resources, Conservation and Recycling

Volume 186,

November 2022

, 106525


Trade contributes to the redistribution of resources among countries and regions. One of the most widely used data sources is the United Nations Commodity Trade Statistics Database (UN Comtrade). Nevertheless, data issues still limit its validity, trustworthiness, and use. A critical issue is the lack of commodity weight information. It relies heavily on data quality to determine the global market's suppliers and consumers. Thus, trade needs reliable methods for filling in the missing physical values. Using statistical approaches, we estimate missing physical values for commodities, countries/areas, and years. The impact of handled data on countries and commodities varies considerably; for example, South Africa's net weight rose by 117% and clocks’ and watches’ by 63% (HS0, 1988–2019), compared with their original data. The directions of net trade flows for 10594 records have been reversed. Finally, the bilateral asymmetry problem improved. Overall, this paper introduces a novel approach for improving data accuracy.


Trade has long been a subject of great interest in various fields (Chenetal.2019). In recent years, there has been increasing interest in evaluating the impact of trade on people's lives and the planet (Dalinetal.2017; Zhangetal.2017). Trade plays a crucial role in redistributing resources and wealth among countries and regions (Xuetal.2020b; Yangetal.2020). For example, trade is vital to circular economy development (Wangetal.2020b). Over the last few decades, resource trade has resulted in a growing share of global resources extraction, reaching 24.8% in 2017 (Figure A1). Therefore, it is necessary to quantify the material flows accompanied by trade, which involves diverse processes.

The United Nations Commodity Trade Information Database (UN Comtrade), established in the early 1960s, is one of the most extensive and accurate international trade statistics databases. For more than 50 years, it has supplied a plethora of trade information to policymakers, business communities, academic institutions, and the general public (Comtrade2019). It stores standardized annual, and monthly trade statistics supplied by countries/areas and reflects detailed international commodity flows between partners, accounting for up to 99% of global merchandise trade (Comtrade2019). Many kinds of research, including our earlier ones, have shown, however, that missing values in UN Comtrade pose statistical issues that can lead to considerable trade misunderstanding (Espinozaand Soulier 2016; Nakajimaetal.2018; Shietal.2021) and the severity of the situation worsens as the proportion of missing values rises. This problem exists for all commodities, countries, and years (Table A1). The lack of data will result in underestimating material flows and environmental influence. However, it will also cause net flow reversals (e.g., shifting from net importers to net exporters). There is thus a pressing need to address the problems associated with missing values in the UN Comtrade database.

In recent years, the proportion of missing values (namely missing monetary values, missing physical values, and both missing, see Figure A2) has increased. The missing weight data is the most common, and in this study, we focus on missing physical values. The following reasons may cause missing physical values: 1) Little focus on trade weight. Custom reports of various countries mainly focus on money rather than weights; 2) Wrong unit conversion (Breweretal.2020a; Breweretal.2020b). Some commodities are not reported in kilograms (kg, for example, natural gas). It may result in errors when converting their unit to kg; 3) The lack of unit conversion. Custom data are given in "quantity"; however, UN Comtrade data are released in "net weight". UN Comtrade may overlook some data when filling out the "net weight", resulting in "false data missing".

Previous studies have used the global average price of a specific commodity or the linear regression method (Dittrichand Bringezu 2010; Dittrichetal.2012; Farhan2015). In addition, UN Statistics Division (UNSD) estimates missing data using the median and unit prices (United Nations Statistics Division September,2017). These methods, however, ignore the varieties between commodities, reporters, and years, limiting their usefulness to the UN Comtrade database. For instance, these methods assume that the price of a car is constant worldwide, which is inconsistent with reality. Furthermore, these methods are inapplicable to custom data reported in kg because trade weight can be determined simply from "quantity".

This paper is the third one in this series on addressing data quality issues of the UN Comtrade database. Our first paper presents the status quo, causes, existing solutions, and challenges of data quality issues (Chenetal.2022). The second paper establishes an improved framework to identify outliers (Jiangetal.2022). This third paper aims to develop a framework to handle missing physical values in UN Comtrade for all commodities from 1988-to 2019. We also quantify the data quality improvements to examine the influence of missing physical values. The rest of this paper is structured as follows. The second section summarizes the primary methods used in this study. The third section presents the results and a brief critique of the findings. The fourth section compares these methods and discusses the limitations of this research. The fifth section highlights the main conclusions of this study.

Section snippets

The classification of data and data miss

UN Comtrade data includes statistics from the original unit of measure (as indicated in Table1). Missing values include missing monetary value, missing physical value, and both missing. The missing monetary values are primarily the result of omissions; however, the reasons for missing physical values are more complicated (already introduced in the first section). This study only shows the results of missing physical values. With the same framework, missing monetary values can be handled


In the Material Flow Analysis (MFA) research, the trade records and the trade weight are very important factors. Trade records can characterize the network architecture developed by countries using graph theory. Trade weight can reflect the complex changes in material flow between countries. Firstly, this section shows the distribution of the missing values by time, country, and commodity, which demonstrates the change in the trade records compared to discarding missing physical values

Method comparison

Under most circumstances, one method is challenging to adapt to all countries and commodities. As a result, we compare the method performance by commodities and countries. All method performance results are available on figshare (Zhangetal.2022).

The method selection of different commodities in the same country is different based on the performance evaluation results. Here we take China as an example since this pattern can be identified in other countries. As shown in Fig.6, Method 1 works


This paper presented a framework consisting of seven methods that can be used to deal with the missing values of all commodities in the UN Comtrade database. One of the main advantages of our framework is that it is based on the estimation of the data distribution, which can better distinguish the heterogeneity of all commodities in different countries and different years, which increases the accuracy of the estimation. These estimation procedures significantly improved the data quality of the

CRediT authorship contribution statement

Zhihe Zhang: Resources, Investigation, Methodology, Validation, Formal analysis, Writing – original draft. Zhihan Jiang: Investigation, Methodology. Chuke Chen: Resources, Formal analysis, Visualization, Writing – review & editing. Xu Zhang: Writing – review & editing. Heming Wang: Conceptualization, Supervision, Writing – review & editing, Funding acquisition, Methodology, Formal analysis. Nan Li: Conceptualization, Supervision, Writing – review & editing. Peng Wang: Methodology, Writing –

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.


This research was supported by the National Natural Science Foundation of China (No.41871204, No. 71961147003, No. 52170184, and No.52070034).

References (33)

  • Wei-Qiang Chen et al.Sustainable cycles and management of plastics: a brief review of RCR publications in 2019 and early 2020

    Resour., Conserv. Recycl.


  • B. Chen et al.Global energy flows embodied in international trade: a combination of environmentally extended input–output analysis and complex network analysis

    Appl. Energy


  • Brewer, T.D.; Abbott, D.; Lal, N.; Sharp, M.; am Thow; Andrew, N.L. (2020a): A method for cleaning trade data for...
  • Brewer, T.D.; Andrew, N.L.; Sharp, M.K.; am Thow; Kottage, H.; Jones, S. (2020b): A method for cleaning trade data for...
  • Chuke Chen et al.

    Advancing Un Comtrade for physical trade flow analysis: review of data quality issues and solutions

    SSRN J.


  • Comtrade, U.N. (2019): UN Comtrade. In United Nations Commodity Trade Statistics...
  • Cited by (1)

    • Advancing UN Comtrade for Physical Trade Flow Analysis: Review of Data Quality Issues and Solutions

      2022, Resources, Conservation and Recycling

      International trade has been considered a critical driving force of material flows and their environmental pressures, which has been a global research hotspot. The United Nations Commodity Trade Statistics Database (UN Comtrade) is the original and probably the most widely-used data source to support the physical trade analysis. However, data discrepancies have been discovered in UN Comtrade, which may lead to diametrically conflicted conclusions if not properly addressed. To promote applications of UN Comtrade, this article reviews data statistics criteria and preprocessing procedures, discusses three main data quality issues (outliers, missing values, and bilateral asymmetries), and reviews methods to explore adequate options. It is revealed that data quality issues existed in data of almost all the commodities, reporters, and periods, but existing methods are subject to certain limitations. Furthermore, this article presents a brief introduction of our following work on addressing these issues.

    (Video) Trade misinvoicing, Method#1, Partner Country Method (PCM) + (Case study)

    Recommended articles (6)

    • Research article

      Examining the trade-offs in potential retail benefits of different expiration date modes: Insights into multidimensional scenarios

      Resources, Conservation and Recycling, Volume 186, 2022, Article 106511

      A significant challenge in today's food system is variable and unknown food quality affected by various temperatures, resulting in unreliable fixed expiration dates (FED) printed on food packaging. The dynamic expiration date (DED) mode is proposed as an alternative to the FED mode; however, practitioners must guard against some major uncertainties when setting expiration dates (shelf life) for fresh food. As such, we firstly developed a computer-simulated experiment based on a shelf life prediction model for multiple scenarios (i.e., fresh food's initial quality, quality change rate, and shelf temperature) to evaluate potential retail benefits of both date modes. The results show that the DED mode is a promising approach; yet a significant difference in retail benefits emerged between the DED and FED modes. On the one hand, the DED mode could reduce food waste by 10.02% at low temperatures (0−8 °C), but it may increase food waste by 15.94% and 31.71% under the temperature abuse conditions of 12−18 °C and 20−28 °C, respectively. On the other hand, the expired food sold rate within the FED mode is influenced by shelf temperatures, quality change rate, and initial quality of fresh foods. A one-at-a-time perturbation analysis finds that fresh food's shelf life associated with food quality is particularly sensitive to the food waste rate and expired food sold rate in both date modes. This study supports the trade-offs in retail benefits between the two modes while encouraging retailers to proactively optimize DED use.

    • Research article

      Quantification and spatial distribution of aluminum in-use stocks in Henan Province, China

      Resources, Conservation and Recycling, Volume 186, 2022, Article 106540

      The accounting of aluminum in-use stocks (ALIS) has received widespread attention, but the systematic assessment of China's ALIS at the provincial level remains unclear. Based on a bottom-up approach, this study utilizes Henan Province as an example to investigate the ALIS and its spatial patterns at China's provincial level. In 2017, the total and per capita of ALIS in Henan were 12Tg and 125kg/capita. About 90% of the ALIS deposited in residential buildings, nonresidential buildings, automobiles, power generation, power transmission and distribution, and household durables. An apparent difference was revealed between urban and rural ALIS, showing that the total reserves of the building and consumer durables sectors in urban are about 70% higher than in rural areas. At the prefecture-level, the aluminum stock density varied from 35 to 225t/km2, and the per capita ALIS ranged from 96 to 227kg/capita, indicating a significant imbalance in spatial distribution.

    • Research article

      Carbon footprint and voting preferences of a council

      Resources, Conservation and Recycling, Volume 186, 2022, Article 106535

      (Video) [Panel] An initial assessment of mismanaged Dutch Plastic Waste flows and three other talks!

      Accounting for carbon should be undertaken at multiple scales to create awareness of the negative environmental impacts of consumption. We undertake a comprehensive consumption-based supply-chain assessment of a community's emissions for a selected council area in the Greater Sydney region of Australia using multi-regional input-output analysis, by constructing a customised input-output table with data from the Australian Household Expenditure Survey on items related to food, beverages, housing, transport, energy, clothing & footwear, household appliances & furniture, medical services, communication, recreation and education. We quantify the Scope-1, -2, and -3 emissions of households in the council area, and put the results in the context of voting preferences of the Council community. Our results suggest that despite left-wing voter preferences, the consumption habits of green voters carry high carbon footprints – the per-capita carbon footprint of Inner West residents is about 1.3 times more than the footprint of residents in the Rest of Greater Sydney. Furthermore, about two-thirds of the footprint is embodied in upstream supply chains for satisfying consumption. This result is significant in that it means that if policy addressed only energy-related direct emissions, it would be missing a majority of the population's CO2e footprint.

    • Research article

      Comparing potential cost savings of energy quota trading and carbon emissions trading for China's industrial sectors

      Resources, Conservation and Recycling, Volume 186, 2022, Article 106544

      In response to the severity of its environmental problems, China is fully promoting the construction of a national energy quota trading (EQT) market and a national carbon emission trading scheme (ETS). However, there is currently a lack of studies comparing the cost savings of EQT and ETS from the perspective of different industries in different provinces under the same environmental regulation. To fill this gap, this paper first estimates the shadow price of energy and CO2 in 30 Chinese provinces' 38 industrial sectors using provincial-industrial level data from 2005 to 2016 based on the duality theory of non-radial directional distance function. Thus, at the methodological level, this study also overcomes the shortcomings of the DEA method used in most previous studies, that is, it is not appropriate for the cross-industry and cross-province heterogeneity characteristics of the research target. It then evaluates the cost savings of EQT and ETS by simulating actual trade scenarios. We find that the single EQT market obtains greater cost-saving benefits than the mixed market and the single ETS market with the same environmental target. We suggest that the energy consumption units should choose the trading market based on their energy-saving and emissions reduction costs, which is more conducive to achieving the set environmental goals and obtaining cost savings, taking into account different regional industry structures, natural resources, and environmental regulation intensity.

    • Research article

      Beneficiation of Nickel from Ultramafic Ores: Using Sodium Citrate as a Green Processing Reagent

      Resources, Conservation and Recycling, Volume 186, 2022, Article 106496

      The processing of ultramafic ores requires fine grinding, which can result in slime-coating of the nickel-bearing mineral, pentlandite, by the phyllosilicate gangue mineral, serpentine. This lowers nickel recovery and concentrate grade during froth flotation operations. The presence of divalent cations in process water exacerbates the detrimental effects of phyllosilicates on nickel recovery by enhancing the positive surface charge of serpentine and causing charge reversal of pentlandite. To mitigate these issues, an environmentally benign, nontoxic, and biodegradable reagent, sodium citrate, was used in this study to chelate divalent cations on the pentlandite surface and desorb them from the mineral surface. The metal-citrate complexes adsorb onto the positively charged serpentine surface, reversing its charge. Sodium citrate was able to restore the surface charge and hydrophobicity of pentlandite, reverse the charge of serpentine, desorb the divalent cations from the mineral surface, and prevent MgO gangue recovery to the froth phase. Microflotation studies demonstrated that upon the addition of sodium citrate, nickel recovery increased from 74 to 96% and nickel grade was improved from 11 to 15.1% . The hydrophobicity of pentlandite was restored upon conditioning with citrate as evident by increase in water static contact angle from 50° to 70° , DLVO interactions showed that addition of citrate prevents slime coating of pentlandite by serpentine. The results demonstrated the depressive and dispersive action of sodium citrate.

    • Research article

      Enhanced resource recovery from wastewater using electrochemical-osmotic system with nanofiltration membranes

      Resources, Conservation and Recycling, Volume 186, 2022, Article 106555

      Mining diverse resources from electroplating wastewater are critical for maximizing wastewater economic value and minimizing adverse environmental impacts. We have demonstrated an autonomous electrochemical-osmotic system (EOS), which can simultaneously recover copper metals, water, and electric energy from electroplating wastewater. In this study, we enhanced water production, electricity generation, and metal recovery of the EOS using polyelectrolyte multilayer nanofiltration (PMNF) membranes instead of the forward osmosis membranes. The PMNF membrane is fabricated by alternately depositing two oppositely charged polymers, poly(diallyldimethylammonium chloride) (PDADMAC, (+)) and poly(sodium 4-styrenesulfonate) (PSS, (−)), on the nanofiltration (NF) membrane substrate. Results suggested that the water/salt selectivity of the PMNF membrane increase to 25.1L mol−1 with the increase of the number of the polyelectrolyte deposited layers to 6.0. The EOS assembled with a 1.5-layer PMNF membrane achieved a water production rate of 6.06L m−2 h−1 and a maximum power density of 1.18mW cm−2 by treating synthetic electroplating wastewater, 2.63 and 1.21 times higher than conventional EOS (i.e., FO membrane), respectively. Further, we clarified that the PMNF-EOS effectively inhibited the forward transport of various metal cations (e.g., Cu2+, Cr6+, Zn2+, and Ni2+) while also extracting freshwater, generating power, and recovering copper from electroplating wastewater. Our study provided a new avenue for developing a high-performance EOS to reclaim multiple resources from electroplating wastewater.

    View full text

    © 2022 Elsevier B.V. All rights reserved.

    (Video) Automate, Analyse, Achieve: Trading Technology Today


    How can I get data from UN Comtrade? ›

    Just type the commodity text on the box, select reporter/partner, and click Search. The system will search for the latest available data. The basic selection option is a little more sophisticated but still user-friendly. You can search for commodities within the commodities tab.

    What is the UN Comtrade database? ›

    UN Comtrade Database

    UN Comtrade is a repository of official international trade statistics and relevant analytical tables. All data is accessible through API. Weibull International Trade Online Dashboard.

    Where can I get trade data? ›

    Other U.S. Government sources for trade data include: Census' USA Trade Online (goods trade), USITC's Dataweb (goods trade), and the Bureau of Economic Analysis' International Transaction Accounts (goods and services trade).

    What is reporter and partner in UN Comtrade? ›

    A: "Reporter" is the country (or area) which reports the trade data to us. A "Partner" is the country or area from/to which the commodity was imported or exported, as declared by the reporting country.

    What is international trade in business? ›

    international trade, economic transactions that are made between countries. Among the items commonly traded are consumer goods, such as television sets and clothing; capital goods, such as machinery; and raw materials and food.

    What is an export commodity? ›

    Export commodities are raw materials, goods or resources produced in one country, and shipped to other countries to be distributed and sold.


    1. Trade Methodology and International Marketing
    2. Report Briefing: Reckoning with the U.S. Role in Global Ocean Plastic Waste
    (National Academies - Earth and Life Studies)
    3. SafeHaven (SHA) | Crypto First-Look Fundamentals
    4. How to Find an Edge in the Stock Market, with Dennis Wilborn (TMC Show)
    (Mara Wealth)
    5. IBA's Asset Management Webinar
    6. Crypto & Stock Trading Community Show - 11.07.2021 - The Broiler Chickens Show
    (The Rational Investor)

    Top Articles

    Latest Posts

    Article information

    Author: Jeremiah Abshire

    Last Updated: 12/28/2022

    Views: 6143

    Rating: 4.3 / 5 (74 voted)

    Reviews: 89% of readers found this page helpful

    Author information

    Name: Jeremiah Abshire

    Birthday: 1993-09-14

    Address: Apt. 425 92748 Jannie Centers, Port Nikitaville, VT 82110

    Phone: +8096210939894

    Job: Lead Healthcare Manager

    Hobby: Watching movies, Watching movies, Knapping, LARPing, Coffee roasting, Lacemaking, Gaming

    Introduction: My name is Jeremiah Abshire, I am a outstanding, kind, clever, hilarious, curious, hilarious, outstanding person who loves writing and wants to share my knowledge and understanding with you.