English
Neural Networks

구축 방법 — Neural Networks · v1.0 · 2026-10-27

■ 방법

  • 탭 4개: Neurons & Layers / Backpropagation / Training Dynamics / Convolution. 탭마다 개요·체험·요약·참고문헌
  • 계산은 모두 브라우저에서: 직접 쓴 JavaScript(nn-core.js), 외부 라이브러리 없음. 다층 퍼셉트론·역전파·SGD/Momentum/Adam·합성곱을 직접 구현. 사전 학습한 모델 없음
  • 신경망: 완전연결, 은닉 활성 tanh·ReLU·sigmoid·선형 중 고름, 출력 sigmoid 1개, 이진 교차 엔트로피. 초기화 정규분포 × √(1/n_in)(ReLU는 √(2/n_in)), 편향 0, 시드 고정
  • Neurons & Layers: 자료 300점(곧은 경계·XOR·고리·두 개의 달·나선, 시드 11), 새 점 300개(시드 12)로 정확도. 은닉층 0–3, 층마다 1–16 뉴런. Adam 학습률 0.02, 묶음 32, 1,500단계 (모두 정한 값)
  • Backpropagation: 2-2-2 sigmoid, 오차 ½Σ(target − out)², 입력·목표·가중치·편향 시작값은 Mazur(2015) 예제와 같음, 편향 고정. 기울기마다 중앙 차분(h 1e-5) 수치 미분을 같이 보여 줌
  • Training Dynamics: 나선 300점, [2, 16, 16, 1] tanh, 세 옵티마이저가 같은 시작 가중치(시드 5)와 단계마다 같은 묶음(시드 31 + 단계)을 씀. 1,000단계, 20단계마다 전체 손실. Momentum 0.9, Adam β 0.9·0.999, ε 1e-8. 손실이 20을 넘거나 수가 아니면 '발산'으로 멈춤
  • Convolution: 3×3 'valid' 상관(딥러닝 관례, 커널을 뒤집지 않음), 선택으로 ReLU, 2×2 max pooling. 음수가 있으면 파랑(+)·주황(−)으로, 없으면 회색으로 최대값에 맞춰 그림
  • LLM 없음

■ 데이터

  • 2차원 점 자료 5가지: 코드로 생성(nn-core.js dataset), 성격 합성
  • 이미지: scikit-image data.chelsea(사진 Stefan van der Walt) · CC0 · 회색조로 바꾸고 180×120으로 줄임(src/topics/neural-networks/data/chelsea-gray.png)

■ 검증

  • tests/neural-networks/core-test.mjs: Mazur(2015) 예제 대조 — 처음 오차 0.298371109, η 0.5 한 번 갱신 뒤 w5 0.35891648 등 공개된 값과 일치(오차 값은 예제가 중간값을 반올림해 소수 일곱째 자리부터 다름)
  • MLP 기울기 검사(중앙 차분) tanh·ReLU·sigmoid 모두 통과
  • 학습 확인: XOR·고리·나선이 51–55%에서 100%로. 나선, 같은 조건 1,000단계: SGD η 0.01 손실 0.663, Momentum 0.586, Adam 0.098(정확도 97.7%) — 화면과 Node 결과가 같음
  • 합성곱·max pooling을 손계산 예와 대조
  • 화면: Playwright로 375·1920px × 한·영 × 탭마다 콘솔 오류·넘침·빈 그림·남은 한글 검사

■ 범위

  • 원리를 보이는 작은 신경망이고 실제 모델 성능을 뜻하지 않음. 실제 합성곱 신경망은 커널 값을 학습하지만 여기서는 사람이 정하거나 입력함

■ 출처

  • Rumelhart, D.E., Hinton, G.E., Williams, R.J. (1986). Learning representations by back-propagating errors. Nature 323
  • Mazur, M. (2015). A step by step backpropagation example. mattmazur.com
  • Kingma, D.P., Ba, J. (2015). Adam: a method for stochastic optimization. ICLR
  • Glorot, X., Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. AISTATS
  • LeCun, Y. et al. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86(11)
  • Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. MIT Press
  • van der Walt, S. et al. (2014). scikit-image: image processing in Python. PeerJ 2

■ 이력

  • 2026-10-27 · v1.0 · 공개

How it's built — Neural Networks · v1.0 · 2026-10-27

■ Method

  • 4 tabs: Neurons & Layers / Backpropagation / Training Dynamics / Convolution. Each tab has an overview, hands-on part, summary and references
  • Everything is computed in the browser with hand-written JavaScript (nn-core.js), no external libraries: multilayer perceptron, backpropagation, SGD/Momentum/Adam and convolution are implemented directly. No pre-trained models
  • Network: fully connected; hidden activation tanh, ReLU, sigmoid or linear; one sigmoid output; binary cross-entropy. Initialisation normal × √(1/n_in) (√(2/n_in) for ReLU), zero biases, fixed seeds
  • Neurons & Layers: 300 points (straight split, XOR, ring, two moons, spiral; seed 11), accuracy on 300 new points (seed 12). 0–3 hidden layers, 1–16 neurons each. Adam, learning rate 0.02, batch 32, 1,500 steps (all fixed choices)
  • Backpropagation: 2-2-2 sigmoid network, error ½Σ(target − out)², inputs, targets, weights and biases start from the Mazur (2015) example; biases held fixed. Each gradient is shown next to a central-difference numerical derivative (h 1e-5)
  • Training Dynamics: spiral, 300 points, [2, 16, 16, 1] tanh. The three optimizers share starting weights (seed 5) and the same batch at each step (seed 31 + step). 1,000 steps, full loss every 20 steps. Momentum 0.9, Adam β 0.9/0.999, ε 1e-8. A run stops as 'diverged' when the loss exceeds 20 or is not a number
  • Convolution: 3×3 'valid' cross-correlation (the deep-learning convention; the kernel is not flipped), optional ReLU, 2×2 max pooling. Outputs with negative values are drawn blue (+) / orange (−), otherwise grey scaled to the maximum
  • No LLM

■ Data

  • Five 2-D point sets generated in code (nn-core.js dataset); synthetic
  • Image: scikit-image data.chelsea (photo by Stefan van der Walt) · CC0 · converted to greyscale and reduced to 180×120 (src/topics/neural-networks/data/chelsea-gray.png)

■ Validation

  • tests/neural-networks/core-test.mjs: checked against the Mazur (2015) example — starting error 0.298371109, w5 0.35891648 after one update with η 0.5, and the other published values (the error after one step differs from the 7th decimal because the example rounds intermediate values)
  • MLP gradient check (central differences) passes for tanh, ReLU and sigmoid
  • Training: XOR, ring and spiral go from 51–55% to 100%. Spiral, same settings, 1,000 steps: SGD η 0.01 loss 0.663, Momentum 0.586, Adam 0.098 (accuracy 97.7%) — identical in the browser and in Node
  • Convolution and max pooling checked against hand-computed examples
  • Screens: Playwright at 375 and 1920 px × Korean/English × every tab, checking console errors, overflow, blank plots and leftover Korean

■ Scope

  • Small networks that show principles; results say nothing about real model performance. Real convolutional networks learn their kernels; here they are chosen or typed by the user

■ Sources

  • Rumelhart, D.E., Hinton, G.E., Williams, R.J. (1986). Learning representations by back-propagating errors. Nature 323
  • Mazur, M. (2015). A step by step backpropagation example. mattmazur.com
  • Kingma, D.P., Ba, J. (2015). Adam: a method for stochastic optimization. ICLR
  • Glorot, X., Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. AISTATS
  • LeCun, Y. et al. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86(11)
  • Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. MIT Press
  • van der Walt, S. et al. (2014). scikit-image: image processing in Python. PeerJ 2

■ History

  • 2026-10-27 · v1.0 · Released