Skip to content

Commit 7da3ce0

Browse files
๐ŸŒ [i18n-KO] Translated bertology.mdx to Korean (#23968)
* docs: ko: `bertology.mdx` * feat: nmt draft * fix: manual edits * fix: resolve suggestions Co-authored-by: Hyeonseo Yun <[email protected]> --------- Co-authored-by: Hyeonseo Yun <[email protected]>
1 parent c938597 commit 7da3ce0

File tree

2 files changed

+39
-2
lines changed

2 files changed

+39
-2
lines changed

โ€Ždocs/source/ko/_toctree.ymlโ€Ž

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -168,8 +168,8 @@
168168
title: (๋ฒˆ์—ญ์ค‘) Attention mechanisms
169169
- local: pad_truncation
170170
title: ํŒจ๋”ฉ๊ณผ ์ž˜๋ผ๋‚ด๊ธฐ
171-
- local: in_translation
172-
title: (๋ฒˆ์—ญ์ค‘) BERTology
171+
- local: bertology
172+
title: BERTology
173173
- local: in_translation
174174
title: (๋ฒˆ์—ญ์ค‘) Perplexity of fixed-length models
175175
- local: in_translation
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# BERTology
14+
15+
BERT์™€ ๊ฐ™์€ ๋Œ€๊ทœ๋ชจ ํŠธ๋žœ์Šคํฌ๋จธ์˜ ๋‚ด๋ถ€ ๋™์ž‘์„ ์กฐ์‚ฌํ•˜๋Š” ์—ฐ๊ตฌ ๋ถ„์•ผ๊ฐ€ ์ ์  ๋” ์ค‘์š”ํ•ด์ง€๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
16+
ํ˜น์ž๋Š” "BERTology"๋ผ ์นญํ•˜๊ธฐ๋„ ํ•ฉ๋‹ˆ๋‹ค. ์ด ๋ถ„์•ผ์˜ ์ข‹์€ ์˜ˆ์‹œ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค:
17+
18+
19+
- BERT๋Š” ๊ณ ์ „์ ์ธ NLP ํŒŒ์ดํ”„๋ผ์ธ์˜ ์žฌ๋ฐœ๊ฒฌ - Ian Tenney, Dipanjan Das, Ellie Pavlick:
20+
https://arxiv.org/abs/1905.05950
21+
- 16๊ฐœ์˜ ํ—ค๋“œ๊ฐ€ ์ •๋ง๋กœ 1๊ฐœ๋ณด๋‹ค ๋‚˜์€๊ฐ€? - Paul Michel, Omer Levy, Graham Neubig:
22+
https://arxiv.org/abs/1905.10650
23+
- BERT๋Š” ๋ฌด์—‡์„ ๋ณด๋Š”๊ฐ€? BERT์˜ ์–ดํ…์…˜ ๋ถ„์„ - Kevin Clark, Urvashi Khandelwal, Omer Levy, Christopher D. Manning:
24+
https://arxiv.org/abs/1906.04341
25+
- CAT-probing: ํ”„๋กœ๊ทธ๋ž˜๋ฐ ์–ธ์–ด์— ๋Œ€ํ•ด ์‚ฌ์ „ํ›ˆ๋ จ๋œ ๋ชจ๋ธ์ด ์–ด๋–ป๊ฒŒ ์ฝ”๋“œ ๊ตฌ์กฐ๋ฅผ ๋ณด๋Š”์ง€ ์•Œ์•„๋ณด๊ธฐ ์œ„ํ•œ ๋ฉ”ํŠธ๋ฆญ ๊ธฐ๋ฐ˜ ์ ‘๊ทผ ๋ฐฉ๋ฒ•:
26+
https://arxiv.org/abs/2210.04633
27+
28+
์šฐ๋ฆฌ๋Š” ์ด ์ƒˆ๋กœ์šด ์—ฐ๊ตฌ ๋ถ„์•ผ์˜ ๋ฐœ์ „์„ ๋•๊ธฐ ์œ„ํ•ด, BERT/GPT/GPT-2 ๋ชจ๋ธ์— ๋‚ด๋ถ€ ํ‘œํ˜„์„ ์‚ดํŽด๋ณผ ์ˆ˜ ์žˆ๋Š” ๋ช‡ ๊ฐ€์ง€ ๊ธฐ๋Šฅ์„ ์ถ”๊ฐ€ํ–ˆ์Šต๋‹ˆ๋‹ค.
29+
์ด ๊ธฐ๋Šฅ๋“ค์€ ์ฃผ๋กœ Paul Michel์˜ ํ›Œ๋ฅญํ•œ ์ž‘์—…์„ ์ฐธ๊ณ ํ•˜์—ฌ ๊ฐœ๋ฐœ๋˜์—ˆ์Šต๋‹ˆ๋‹ค
30+
(https://arxiv.org/abs/1905.10650):
31+
32+
33+
- BERT/GPT/GPT-2์˜ ๋ชจ๋“  ์€๋‹‰ ์ƒํƒœ์— ์ ‘๊ทผํ•˜๊ธฐ,
34+
- BERT/GPT/GPT-2์˜ ๊ฐ ํ—ค๋“œ์˜ ๋ชจ๋“  ์–ดํ…์…˜ ๊ฐ€์ค‘์น˜์— ์ ‘๊ทผํ•˜๊ธฐ,
35+
- ํ—ค๋“œ์˜ ์ถœ๋ ฅ ๊ฐ’๊ณผ ๊ทธ๋ž˜๋””์–ธํŠธ๋ฅผ ๊ฒ€์ƒ‰ํ•˜์—ฌ ํ—ค๋“œ ์ค‘์š”๋„ ์ ์ˆ˜๋ฅผ ๊ณ„์‚ฐํ•˜๊ณ  https://arxiv.org/abs/1905.10650์—์„œ ์„ค๋ช…๋œ ๋Œ€๋กœ ํ—ค๋“œ๋ฅผ ์ œ๊ฑฐํ•˜๋Š” ๊ธฐ๋Šฅ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.
36+
37+
์ด๋Ÿฌํ•œ ๊ธฐ๋Šฅ๋“ค์„ ์ดํ•ดํ•˜๊ณ  ์ง์ ‘ ์‚ฌ์šฉํ•ด๋ณผ ์ˆ˜ ์žˆ๋„๋ก [bertology.py](https://github.com/huggingface/transformers/tree/main/examples/research_projects/bertology/run_bertology.py) ์˜ˆ์ œ ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์ถ”๊ฐ€ํ–ˆ์Šต๋‹ˆ๋‹ค. ์ด ์˜ˆ์ œ ์Šคํฌ๋ฆฝํŠธ์—์„œ๋Š” GLUE์— ๋Œ€ํ•ด ์‚ฌ์ „ํ›ˆ๋ จ๋œ ๋ชจ๋ธ์—์„œ ์ •๋ณด๋ฅผ ์ถ”์ถœํ•˜๊ณ  ๋ชจ๋ธ์„ ๊ฐ€์ง€์น˜๊ธฐ(prune)ํ•ด๋ด…๋‹ˆ๋‹ค.

0 commit comments

Comments
ย (0)