TMF-GNN: Temporal matrix factorization-based graph neural network for multivariate time series forecasting with missing values

Suhyeon Kim, Taek-Ho Lee, 이정혜 (2025) · Expert Systems with Applications 275:127001 · DOI ↗

TMF-GNN — 결측 (missing values) 있는 다변량 시계열 (MTS) 예측 의 end-to-end 방법으로, imputation + forecasting 의 2-stage 분리 한계를 해소. 핵심 통찰: 기존 TMF (Yu et al 2016 TRMF, Chen et al 2022 NoTMF) 가 temporal regularization 만 사용 → variable-wise inter-correlation 무시. TMF-GNN 은 (i) Matrix Factorization module (factor matrix $U \in R^{N \times R}$ , $V \in R^{R \times T}$ 의 partial observation reconstruction) + (ii) GCTC regularization (Graph Convolution + Temporal Convolution): Adaptive adjacency $A = f_{sc}(\text{ReLU}(L \cdot ZZ^T))$ + top- $k$ neighbor sparsification + GCN 으로 variable interaction + TCN 으로 temporal pattern + concat → fc → forecast. 4 dataset (GFT 3.24% missing, AQ 14.4%, PM2.5 50.5%, Traffic) × 3 baseline (TMF, TRMF, NoTMF) × 5 metric (MAE, RMSE, MAPE, ND, NRMSE) × 3 horizon (1, 3, 6) 평가. TMF-GNN 이 모든 baseline outperform. 이정혜 의 실타래 5 (Graph-based methods) + 실타래 4 (Representation Learning) 의 MTS forecasting + missing values 응용. Suhyeon Kim 의 4-paper series 의 graph + sequential ML 정점.

RQ: (1) MTS 의 결측값 을 separate imputation stage 없이 어떻게 forecasting 정확도 향상에 활용하는가? (2) 기존 TMF (TRMF, NoTMF) 의 temporal-only regularization 한계 (variable interaction 무시) 를 어떻게 해소하는가? (3) Graph structure 와 temporal pattern 을 end-to-end 로 동시 학습하는 deep learning-based regularization 은?
방법론: MF module (Eq 5): $\mathcal{L}_m = \lambda_1 \|\mathcal{P}_{\Omega}(X - UV)\|_F^2$ , 관찰된 cell 만 reconstruction. GCTC regularization (Eq 8): (i) Adaptive adjacency $A$ 생성 + softmax + top- $k$ neighbor sparsification, (ii) 그래프 합성곱 신경망 (Kipf-Welling 2016) 의 $H^{(l+1)} = f_g(A H^{(l)} W^{(l)} + b^{(l)})$ , (iii) temporal-convolution-network (Wan et al 2019) 의 1D FCN + causal convolution + dilated convolution + residual connection, (iv) Sliding window $\delta$ 의 sequence input $V_t$ , (v) Row-mean aggregator $h_g$ + concat $h_t^v$ + fc layer → forecast $H_t$ , (vi) Graph regularization (smoothness + sparsity). Total loss (Eq 9): $\mathcal{L} = \mathcal{L}_m + \mathcal{L}_r$
데이터: 4 dataset (Table 1) — (i) GFT (Google Flu Trends) 159 variables × 620 weekly samples (2003-2015), 3.24% missing. (ii) AQ (Air Quality) 13 vars × 8,760 hourly (2004-2005, Italy), 14.4% missing. (iii) PM2.5 7 vars × 8,760 hourly (Beijing 2013), 50.5% missing. (iv) Traffic 962 vars × 17,544 hourly (California Bay Area 2015-2016), no missing. 평가: GFT next 12 weeks, AQ/PM2.5/Traffic next 7 days (24 timesteps × 7 windows). 3 horizon (1, 3, 6), 3 trial avg
주요 발견: (i) TMF-GNN > TMF (Xiong et al 2010) > TRMF (Yu et al 2016) > NoTMF (Chen et al 2022) in 모든 5 metric. (ii) Higher missing ratio (PM2.5 50.5%) 에서 우위 더 큼 — adaptive graph 가 missing 영향 mitigation. (iii) Adaptive adjacency 가 prior graph structure 없이도 효과적 — variable inter-correlation 자동 학습. (iv) TCN > AR/VAR for temporal regularization — sequential deep learning 우위. (v) Hyperparameter sensitivity (Appendix A): $\lambda_1, \lambda_2, \lambda_3$ , hidden layers, $k$ , learning rate, epoch
시사점: (a) End-to-end approach 의 2-stage error propagation 회피 — imputation 정확도가 forecasting 정확도를 보장하지 않는다는 system identification 문제 해결. (b) Graph-based MTS forecasting 의 missing values 확장 — traffic flow 외 air quality, healthcare, IoT sensor 의 high-missing domain 적용. (c) Adaptive graph structure 의 자동 학습 — domain knowledge 부재 환경에서 graph prior 불필요. (d) Deep learning regularization 의 TMF 통합 — naive AR/VAR 의 naive time-series model 한계 극복. (e) 환경 모니터링, 의료 시계열, IoT 등 결측 빈번 도메인 의 new baseline

(원본 PDF figure 미추출 — 본문 architecture 식 + Algorithm 1 로 구조 surface)

요약

본 paper 는 이정혜 의 3 기 SNU TEMEP 시기 의 graph + sequential ML 결합 작업. Suhyeon Kim (W2V-LSA, LBC, RSAE-BA 의 동일 제 1 저자, Pyeongchang National University Graduate School of Data Science) + Taek-Ho Lee (SeoulTech Industrial Engineering + TEMEP) + 이정혜 (corresponding, SNU Institute of Engineering Research + Graduate School of Engineering Practice) 의 공저. 동기는 MTS missing values 의 pervasive 문제 — sensor 고장, network failure, data transmission 문제, storage 한계가 IoT/health/environmental MTS 의 chronic challenge. 기존 imputation approach 의 한계:

Statistical imputation (mean/median, AR/MA, KNN, random forest) — separate stage 의 inefficiency
Deep learning imputation (RNN, GAN) — imputation 정확도 ≠ forecasting 정확도
2-stage approach 의 systemic 문제 — error propagation, dual hyperparameter tuning, imputation 이 raw distribution 보존 안 함

기존 TMF approach (Xiong et al 2010, Yu et al 2016 TRMF, Chen et al 2022 NoTMF) 가 imputation 제거 의 alternative — partial observation 만으로 low-rank $U \cdot V$ 분해 + temporal regularization (AR/VAR). 그러나 두 한계: (a) variable interaction 무시 — temporal-only, (b) naive AR/VAR 의 한계. 본 paper 는 graph-based variable interaction + deep learning sequential regularization 으로 동시 해소.

MF module — Reconstruction with missing data. 학습 가능 $U \in R^{N \times R}$ (variable-wise) + $V \in R^{R \times T}$ (temporal). Reconstruction loss (Eq 5):

\mathcal{L}_m = \lambda_1 \|\mathcal{P}_{\Omega}(X - UV)\|_F^2

여기서 $\mathcal{P}_{\Omega}$ = partial observation mask (관찰된 cell 만), $\|\cdot\|_F$ = Frobenius norm. SVD/NMF 같은 complete-data MF 는 missing 불가 — 학습 가능 factor matrix 가 대안.

GCTC regularization — MTS forecasting. 4 sub-module.

(i) Adaptive adjacency $A$ (Eq 6, 7):

A = f_{sc}(\text{ReLU}(L \cdot Z \cdot Z^T)), \quad Z = M W^T

여기서 $M \in R^{N \times K}$ = randomly initialized node embedding, $W \in R^{K \times K}$ = model parameter, $f_{sc}$ = softmax. Top- $k$ neighbor sparsification 으로 noise + computational cost 감소.

(ii) GCN module: $A$ 위에 graph convolution — $H_g = \text{GCN}(A, U)$ . 각 layer:

H^{(l+1)} = f_g(A H^{(l)} W^{(l)} + b^{(l)})

Initial $H^{(0)} = U$ . Variable-wise factor matrix 가 node feature.

(iii) TCN module: Sliding window $\delta$ 의 sequence $V_t = [v_{t-\delta+1}, \ldots, v_t]^T$ . TCN = 1D FCN + causal convolution + dilated convolution + residual connection. Dilated convolution (Eq 3): $F(s) = \sum_{i=0}^{k-1} f(i) \cdot x_{s - d \cdot i}$ .

(iv) Aggregation + forecast: Row-mean aggregator $h_g$ + concat $h_t^v$ + fc layer $f_h$ :

h_t^c = f_h(\text{CONCAT}(h_g, h_t^v))

$H_t$ 가 forecast.

Graph regularization (Chen-Wu-Zaki 2020): $\frac{1}{N^2} \sum_{ij} A_{ij} \|u_i - u_j\|^2 + \beta \|A\|_F^2$ — smoothness + sparsity.

Total loss (Eq 9):

\mathcal{L} = \lambda_1 \|\mathcal{P}_{\Omega}(X - UV)\|_F^2 + \lambda_2 \sum_{t=\delta+1}^{T} \|x_t - H_t\|_F^2 + \lambda_3 \left( \frac{1}{N^2} \sum_{ij} A_{ij} \|u_i - u_j\|^2 + \beta \|A\|_F^2 \right)

Experiments — 4 dataset (GFT, AQ, PM2.5, Traffic) × 3 baseline (TMF, TRMF, NoTMF) × 5 metric × 3 horizon. TMF-GNN 이 모든 dataset + metric + horizon 에서 baseline outperform. 한계: (i) hyperparameter ( $\lambda_1, \lambda_2, \lambda_3$ , $k$ , hidden layer 수, learning rate, epoch) 가 dataset-specific — grid search 필요, (ii) random missingness (MCAR/MAR) 가정 — MNAR (missing not at random) 미검증, (iii) 4 dataset 한정 — 다른 도메인 generalizability, (iv) computational cost (GCN + TCN + MF 동시) 가 큰 dataset 에서 burden. 이정혜 author page 의 실타래 5 (Graph-based methods) 의 발전 + 실타래 4 (Representation Learning) 의 MTS 응용 의 anchor. Suhyeon Kim series 의 4 번째 작품 — text embedding → multistage data mining → autoencoder embedding → graph + sequential 의 progression.

핵심 결과

4 dataset (Table 1)

Dataset	# vars	# samples	Sample rate	Missing ratio
GFT (Google Flu Trends)	159	620	1 week	3.24%
AQ (Air Quality, Italy)	13	8,760	1 hour	14.4%
PM2.5 (Beijing 2013)	7	8,760	1 hour	50.5%
Traffic (California Bay Area)	962	17,544	1 hour	0%

5 evaluation metrics (Eqs 10-14)

Metric	식
MAE	$\frac{1}{
RMSE	$\sqrt{\frac{1}{
MAPE	mean absolute percentage error
ND	normalized deviation
NRMSE	normalized RMSE

Baseline 비교 (TMF, TRMF, NoTMF)

Method	Variable interaction	Temporal regularization
TMF (Xiong et al 2010)	❌	AR
TRMF (Yu et al 2016)	❌	regularized AR (Markov)
NoTMF (Chen et al 2022)	❌	non-temporal MF
TMF-GNN	✅ Adaptive GCN	✅ TCN

정량 결론. TMF-GNN 이 4 dataset × 5 metric × 3 horizon 모두에서 baseline outperform. Adaptive graph learning + TCN deep sequential regularization 의 combined effect. PM2.5 (50.5% missing) 에서 가장 큰 우위 — high missing 환경에서 graph-based variable interaction 의 mitigation 효과 명확.

방법론 노트

MF module reconstruction loss (Eq 5):

\mathcal{L}_m = \lambda_1 \|\mathcal{P}_{\Omega}(X - UV)\|_F^2

여기서 $U \in R^{N \times R}$ = variable-wise factor matrix, $V \in R^{R \times T}$ = temporal factor matrix, $R$ = low-rank dimension, $\mathcal{P}_{\Omega}$ = observation mask (관찰된 cell 만 selection).

Adaptive adjacency (Eq 6, 7):

A = f_{sc}(\text{ReLU}(L \cdot Z \cdot Z^T)), \quad Z = M W^T

A_{kj} = \begin{cases} 1 & A_{kj} \in \text{top-}k(A_k, k_n) \\ 0 & \text{otherwise} \end{cases}

End-to-end 학습 + top- $k$ neighbor sparsification.

GCN layer (Kipf-Welling 2016):

H^{(l+1)} = f_g(A H^{(l)} W^{(l)} + b^{(l)})

TCN dilated convolution (Eq 3):

F(s) = \sum_{i=0}^{k-1} f(i) \cdot x_{s - d \cdot i}

Residual block output (Eq 4): $o = f_g(x + F(x))$ .

Total loss (Eq 9):

\mathcal{L} = \lambda_1 \|\mathcal{P}_{\Omega}(X - UV)\|_F^2 + \lambda_2 \sum_{t=\delta+1}^{T} \|x_t - H_t\|_F^2 + \lambda_3 \left( \frac{1}{N^2} \sum_{ij} A_{ij} \|u_i - u_j\|^2 + \beta \|A\|_F^2 \right)

식별은 (i) low-rank assumption 의 MTS adequacy, (ii) variable interaction graph 가 adaptive learning 으로 식별 가능, (iii) end-to-end joint optimization 의 single loss surface, (iv) sparse adjacency (top- $k$ ) 의 computational tractability. Limitation: hyperparameter dataset-specific + MCAR/MAR 가정 + computational cost.

연구 계보

본 paper 의 MTS forecasting with missing lineage: (i) Imputation approaches — statistical (mean/median, Furnkranz-Falcone 2011), RNN-based (Cao et al 2018, Choi et al 2016 doctorAI, Lipton et al 2016, Yoon et al 2017), GAN-based (Luo et al 2018, 2019; Yoon-Jordon-Schaar 2018 GAIN; Du-Côté-Liu 2023 self-attention), spatiotemporal graph (Marisca-Cini-Alippi 2022). (ii) TMF-based: Xiong et al (2010) original TMF, Yu et al (2016 NeurIPS) TRMF, Chen et al (2022) NoTMF — temporal-only regularization. (iii) MTS forecasting deep learning — RNN variants, TCN (Wan-Mei-Wang-Liu-Yang 2019), Seq2Seq (Sutskever-Vinyals-Le 2014), attention + memory (Fan et al 2019, Yu et al 2020). (iv) Graph-based MTS — Bai-Yao-Li-Wang-Wang (2020), Wu-Pan-Long-Jiang-Zhang (2019, 2020 Graph WaveNet), Bian et al (2022). (v) GCN/GNN: Kipf-Welling (2016 ICLR), Cao-Zhu-Wang-Zhu-Niu (2022), Chami-Abu-El-Haija-Perozzi-Re-Murphy (2020). (vi) TCN: Long-Shelhamer-Darrell (2015 1D FCN), Lea-Flynn-Vidal-Reiter-Hager (2017). (vii) Graph regularization: Chen-Wu-Zaki (2020).

TEMEP 내 sibling: Suhyeon Kim 4-paper series 의 4 번째 (최종) — Word2vec-based latent semantic analysis (W2V-LSA) for topic modeling: A study on blockchain technology trend analysis (text embedding, topic modeling) → A Multi-stage Data Mining Approach for Liquid Bulk Cargo Volume Analysis based on Bill of Lading Data (text + manifold + LSTM, port logistics) → Risk score-embedded deep learning for biological age estimation: Development and validation (autoencoder embedding, medical AI) → 본 paper (graph + sequential + MF, environmental/health MTS). 방법론적 진화: word embedding → manifold + LSTM → autoencoder → GNN + TCN + MF. 이정혜 author page 의 3 기 SNU TEMEP 의 기술경영 + ML methodology 라인의 최근 정점. Taek-Ho Lee (SeoulTech IE + TEMEP affiliation) 의 공저 — TEMEP 내 graph + sequential ML 협업 라인. Wonho Sohn 의 항만 무역 + LSTM 라인 (A Multi-stage Data Mining Approach for Liquid Bulk Cargo Volume Analysis based on Bill of Lading Data 의 sibling) 과 함께 이정혜 3 기 의 MTS forecasting 라인 anchor.