Compare commits
109 Commits
Author | SHA1 | Date |
---|---|---|
Bertrand Benjamin | 83eb9c327b | |
Bertrand Benjamin | ff1ecfef25 | |
Bertrand Benjamin | 921292a0d2 | |
Bertrand Benjamin | 2d08671247 | |
Bertrand Benjamin | a16211cbd4 | |
Bertrand Benjamin | 876f583d51 | |
Bertrand Benjamin | 97b97af2de | |
Bertrand Benjamin | d8d84690c6 | |
Bertrand Benjamin | 18f855ab83 | |
Bertrand Benjamin | 36425e587e | |
Bertrand Benjamin | 8cdeecfc53 | |
Bertrand Benjamin | 1a7c97d869 | |
Bertrand Benjamin | ab5de2711e | |
Bertrand Benjamin | 235019102b | |
Bertrand Benjamin | 8ec24a24b3 | |
Bertrand Benjamin | 2e86b3a0a2 | |
Bertrand Benjamin | 7e6b24aaea | |
Bertrand Benjamin | 6889ddd97c | |
Bertrand Benjamin | 10b9954c05 | |
Bertrand Benjamin | 7553628306 | |
Bertrand Benjamin | effc049578 | |
Bertrand Benjamin | 411f910ce6 | |
Bertrand Benjamin | 00d81d694a | |
Bertrand Benjamin | a8b2ac455d | |
Bertrand Benjamin | 9e0ea14d05 | |
Bertrand Benjamin | 2031ade1ab | |
Bertrand Benjamin | 6ed55c07d4 | |
Bertrand Benjamin | 1d234ea5fc | |
Bertrand Benjamin | 646314ad88 | |
Bertrand Benjamin | 0739cfdae7 | |
Bertrand Benjamin | a50901556e | |
Bertrand Benjamin | abc5513268 | |
Bertrand Benjamin | 598086ddb0 | |
Bertrand Benjamin | 94f8080acd | |
Bertrand Benjamin | 6331573940 | |
Bertrand Benjamin | a292fe23e0 | |
Bertrand Benjamin | 3005d49a1d | |
Bertrand Benjamin | 9fce390304 | |
Bertrand Benjamin | 94f94dae84 | |
Bertrand Benjamin | 596a52b1d0 | |
Bertrand Benjamin | 37c95d75de | |
Bertrand Benjamin | bd91bf51d6 | |
Bertrand Benjamin | c1fd060707 | |
Bertrand Benjamin | a0e94f52b1 | |
Bertrand Benjamin | c84f9845b2 | |
Bertrand Benjamin | d9e95f2186 | |
Bertrand Benjamin | 581b0f4f2f | |
Bertrand Benjamin | 3dbfc85447 | |
Bertrand Benjamin | b5bf1ac137 | |
Bertrand Benjamin | 74d751a586 | |
Bertrand Benjamin | 1855d4016d | |
Bertrand Benjamin | ff94470fb4 | |
Bertrand Benjamin | d322452a6e | |
Bertrand Benjamin | e1d3940e9d | |
Bertrand Benjamin | 7dba11996a | |
Bertrand Benjamin | 3250a600c9 | |
Bertrand Benjamin | 589d63ff29 | |
Bertrand Benjamin | 429fed6a1e | |
Bertrand Benjamin | 1255bf4b9e | |
Bertrand Benjamin | 1fe7665753 | |
Bertrand Benjamin | e08e4a32a8 | |
Bertrand Benjamin | b737612adb | |
Bertrand Benjamin | 9c19e2ac56 | |
Bertrand Benjamin | eb60734c26 | |
Bertrand Benjamin | 329bcc460c | |
Bertrand Benjamin | 95fc842c1d | |
Bertrand Benjamin | e0ca1a458b | |
Bertrand Benjamin | eb1abbe868 | |
Bertrand Benjamin | 412e624791 | |
Bertrand Benjamin | e8bf0b3f0a | |
Bertrand Benjamin | c057fa11e7 | |
Bertrand Benjamin | e15119605f | |
Bertrand Benjamin | 494567cdb5 | |
Bertrand Benjamin | 84fcee625d | |
Bertrand Benjamin | f62c898162 | |
Bertrand Benjamin | 7955b989b4 | |
Bertrand Benjamin | 4f14e3518c | |
Bertrand Benjamin | 4bf8f4003e | |
Bertrand Benjamin | a14d47b15c | |
Bertrand Benjamin | 09ac9f01f8 | |
Bertrand Benjamin | 0a5a931d01 | |
Bertrand Benjamin | 21397272c9 | |
Bertrand Benjamin | 894ebc4ec8 | |
Bertrand Benjamin | f6bfac4144 | |
Bertrand Benjamin | cfd5928853 | |
Bertrand Benjamin | 8fcad94df4 | |
Bertrand Benjamin | 27d7c45980 | |
Bertrand Benjamin | 159e7a9f2e | |
Bertrand Benjamin | 72afb26e2a | |
Bertrand Benjamin | 6eb918e0f5 | |
Bertrand Benjamin | 56a669b2be | |
Bertrand Benjamin | a5f22fc8cd | |
Bertrand Benjamin | 5177df06d7 | |
Bertrand Benjamin | d78fcbc281 | |
Bertrand Benjamin | 98fa768541 | |
Bertrand Benjamin | 00c2681823 | |
Bertrand Benjamin | 52f2f3f4cf | |
Bertrand Benjamin | 4ea7f8db14 | |
Bertrand Benjamin | 04a2506d86 | |
Bertrand Benjamin | 77c358b0c1 | |
Bertrand Benjamin | 1886deb430 | |
Bertrand Benjamin | 5e0f2d92ef | |
Bertrand Benjamin | 49cc52f7d1 | |
Bertrand Benjamin | 6d93ef62d7 | |
Bertrand Benjamin | 488df4cb0c | |
Bertrand Benjamin | 9136f359e0 | |
Bertrand Benjamin | 1dfee17990 | |
Bertrand Benjamin | 400fb0a690 | |
Bertrand Benjamin | 04a1ed9378 |
|
@ -122,3 +122,7 @@ dmypy.json
|
|||
|
||||
# Pyre type checker
|
||||
.pyre/
|
||||
|
||||
# vim
|
||||
.vim
|
||||
|
||||
|
|
26
README.md
26
README.md
|
@ -6,3 +6,29 @@ Cette fois ci, on utilise:
|
|||
- Des fichiers yaml pour les infos sur les élèves
|
||||
- Des notebooks pour l'analyse
|
||||
- Papermill pour produire les notesbooks à partir de template
|
||||
|
||||
## Les fichiers CSV
|
||||
|
||||
les paramètres sont décris dans ./recopytex/config.py
|
||||
|
||||
### Descriptions des questions
|
||||
|
||||
- Trimestre
|
||||
- Nom
|
||||
- Date
|
||||
- Exercice
|
||||
- Question
|
||||
- Competence
|
||||
- Domaine
|
||||
- Commentaire
|
||||
- Bareme
|
||||
- Est_nivele
|
||||
|
||||
|
||||
### Valeurs pour notes les élèves
|
||||
|
||||
- Score: 0, 1, 2, 3
|
||||
- Pas de réponses: .
|
||||
- Absent: a
|
||||
- Dispensé: (vide)
|
||||
|
||||
|
|
|
@ -0,0 +1,5 @@
|
|||
Trimestre,Nom,Date,Exercice,Question,Competence,Domaine,Commentaire,Bareme,Est_nivele,Star Tice,Umberto Dingate,Starlin Crangle,Humbert Bourcq,Gabriella Handyside,Stewart Eaves,Erick Going,Ase Praton,Rollins Planks,Dunstan Sarjant,Stacy Guiton,Ange Stanes,Amabelle Elleton,Darn Broomhall,Dyan Chatto,Keane Rennebach,Nari Paulton,Brandy Wase,Jaclyn Firidolfi,Violette Lockney
|
||||
1,DS,12/01/2021,Exercice 1,1,Calculer,Plop,Coucou,1,1,,,1,0,1,2,3,0,3,3,2,,1,,,,,,,
|
||||
1,DS,12/01/2021,Exercice 1,2,Calculer,C'est trop chouette!,Coucou,1,1,,,1,2,,,3,3,,,,,2,,,,,,,
|
||||
1,DS,12/01/2021,Exercice 1,3,Calculer,Null,Coucou,1,1,,,,3,2,,,,,,,,3,,,,,,,
|
||||
1,DS,12/01/2021,Exercice 1,3,Calculer,Nié,DChic,1,1,,,,2,.,,,,,,,,,,,,,,,
|
|
|
@ -0,0 +1,5 @@
|
|||
Trimestre,Nom,Date,Exercice,Question,Competence,Domaine,Commentaire,Bareme,Est_nivele,Star Tice,Umberto Dingate,Starlin Crangle,Humbert Bourcq,Gabriella Handyside,Stewart Eaves,Erick Going,Ase Praton,Rollins Planks,Dunstan Sarjant,Stacy Guiton,Ange Stanes,Amabelle Elleton,Darn Broomhall,Dyan Chatto,Keane Rennebach,Nari Paulton,Brandy Wase,Jaclyn Firidolfi,Violette Lockney
|
||||
1,DS6,22/01/2021,Exercice 1,Sait pas,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
1,DS6,22/01/2021,Exercice 1,Ha,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
1,DS6,22/01/2021,Exercice 1,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
1,DS6,22/01/2021,Exercice 2,grr,,,,,,,,,,,,,,,,,,,,,,,,,
|
|
|
@ -0,0 +1,13 @@
|
|||
---
|
||||
source: ./
|
||||
output: ./
|
||||
templates: templates/
|
||||
|
||||
tribes:
|
||||
Tribe1:
|
||||
name: Tribe1
|
||||
type: Type1
|
||||
students: tribe1.csv
|
||||
Tribe2:
|
||||
name: Tribe2
|
||||
students: tribe2.csv
|
|
@ -0,0 +1,21 @@
|
|||
Nom,email
|
||||
Star Tice,stice0@jalbum.net
|
||||
Umberto Dingate,udingate1@tumblr.com
|
||||
Starlin Crangle,scrangle2@wufoo.com
|
||||
Humbert Bourcq,hbourcq3@g.co
|
||||
Gabriella Handyside,ghandyside4@patch.com
|
||||
Stewart Eaves,seaves5@ycombinator.com
|
||||
Erick Going,egoing6@va.gov
|
||||
Ase Praton,apraton7@va.gov
|
||||
Rollins Planks,rplanks8@delicious.com
|
||||
Dunstan Sarjant,dsarjant9@naver.com
|
||||
Stacy Guiton,sguitona@themeforest.net
|
||||
Ange Stanes,astanesb@marriott.com
|
||||
Amabelle Elleton,aelletonc@squidoo.com
|
||||
Darn Broomhall,dbroomhalld@cisco.com
|
||||
Dyan Chatto,dchattoe@npr.org
|
||||
Keane Rennebach,krennebachf@dot.gov
|
||||
Nari Paulton,npaultong@gov.uk
|
||||
Brandy Wase,bwaseh@ftc.gov
|
||||
Jaclyn Firidolfi,jfiridolfii@reuters.com
|
||||
Violette Lockney,vlockneyj@chron.com
|
|
|
@ -0,0 +1,21 @@
|
|||
Nom,email
|
||||
Elle McKintosh,emckintosh0@1und1.de
|
||||
Ty Megany,tmegany1@reuters.com
|
||||
Pippa Borrows,pborrows2@a8.net
|
||||
Sonny Eskrick,seskrick3@123-reg.co.uk
|
||||
Mollee Britch,mbritch4@usda.gov
|
||||
Ingram Plaistowe,iplaistowe5@purevolume.com
|
||||
Fay Vanyard,fvanyard6@sbwire.com
|
||||
Nancy Rase,nrase7@omniture.com
|
||||
Rachael Ruxton,rruxton8@bravesites.com
|
||||
Tallie Rushmer,trushmer9@home.pl
|
||||
Seward MacIlhagga,smacilhaggaa@hatena.ne.jp
|
||||
Lizette Searl,lsearlb@list-manage.com
|
||||
Talya Mannagh,tmannaghc@webnode.com
|
||||
Jordan Witherbed,jwitherbedd@unesco.org
|
||||
Reagan Botcherby,rbotcherbye@scientificamerican.com
|
||||
Libbie Shoulder,lshoulderf@desdev.cn
|
||||
Abner Khomich,akhomichg@youtube.com
|
||||
Zollie Kitman,zkitmanh@forbes.com
|
||||
Fiorenze Durden,fdurdeni@feedburner.com
|
||||
Kevyn Race,kracej@seattletimes.com
|
|
|
@ -1,4 +0,0 @@
|
|||
---
|
||||
source: sheets/
|
||||
output: reports/
|
||||
templates: templates/
|
|
@ -1,5 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from .csv_extraction import flat_df_students, flat_df_for
|
||||
from .df_marks_manip import pp_q_scores
|
|
@ -1,30 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
NO_ST_COLUMNS = {
|
||||
"term": "Trimestre",
|
||||
"assessment": "Nom",
|
||||
"date": "Date",
|
||||
"exercise": "Exercice",
|
||||
"question": "Question",
|
||||
"competence": "Competence",
|
||||
"theme": "Domaine",
|
||||
"comment": "Commentaire",
|
||||
"score_rate": "Bareme",
|
||||
"is_leveled": "Est_nivele",
|
||||
}
|
||||
|
||||
COLUMNS = {
|
||||
**NO_ST_COLUMNS,
|
||||
"student": "Eleve",
|
||||
"score": "Score",
|
||||
"mark": "Note",
|
||||
"level": "Niveau",
|
||||
"normalized": "Normalise",
|
||||
}
|
||||
|
||||
VALIDSCORE = {
|
||||
"NOTFILLED": "", # The item is not scored yet
|
||||
"NOANSWER": ".", # Student gives no answer (this score will impact the fianl mark)
|
||||
"ABS": "a", # Student has absent (this score won't be impact the final mark)
|
||||
}
|
|
@ -1,119 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
""" Extracting data from xlsx files """
|
||||
|
||||
import pandas as pd
|
||||
from .config import NO_ST_COLUMNS, COLUMNS, VALIDSCORE
|
||||
|
||||
pd.set_option("Precision", 2)
|
||||
|
||||
|
||||
def try_replace(x, old, new):
|
||||
try:
|
||||
return str(x).replace(old, new)
|
||||
except ValueError:
|
||||
return x
|
||||
|
||||
|
||||
def extract_students(df, no_student_columns=NO_ST_COLUMNS.values()):
|
||||
""" Extract the list of students from df
|
||||
|
||||
:param df: the dataframe
|
||||
:param no_student_columns: columns that are not students
|
||||
:return: list of students
|
||||
"""
|
||||
students = df.columns.difference(no_student_columns)
|
||||
return students
|
||||
|
||||
|
||||
def flat_df_students(
|
||||
df, no_student_columns=NO_ST_COLUMNS.values(), postprocessing=True
|
||||
):
|
||||
""" Flat the dataframe by returning a dataframe with on student on each line
|
||||
|
||||
:param df: the dataframe (one row per questions)
|
||||
:param no_student_columns: columns that are not students
|
||||
:return: dataframe with one row per questions and students
|
||||
|
||||
Columns of csv files:
|
||||
|
||||
- NO_ST_COLUMNS meta data on questions
|
||||
- one for each students
|
||||
|
||||
This function flat student's columns to "student" and "score"
|
||||
"""
|
||||
students = extract_students(df, no_student_columns)
|
||||
scores = []
|
||||
for st in students:
|
||||
scores.append(
|
||||
pd.melt(
|
||||
df,
|
||||
id_vars=no_student_columns,
|
||||
value_vars=st,
|
||||
var_name=COLUMNS["student"],
|
||||
value_name=COLUMNS["score"],
|
||||
).dropna(subset=[COLUMNS["score"]])
|
||||
)
|
||||
if postprocessing:
|
||||
return postprocess(pd.concat(scores))
|
||||
return pd.concat(scores)
|
||||
|
||||
|
||||
def flat_df_for(
|
||||
df, student, no_student_columns=NO_ST_COLUMNS.values(), postprocessing=True
|
||||
):
|
||||
""" Extract the data only for one student
|
||||
|
||||
:param df: the dataframe (one row per questions)
|
||||
:param no_student_columns: columns that are not students
|
||||
:return: dataframe with one row per questions and students
|
||||
|
||||
Columns of csv files:
|
||||
|
||||
- NO_ST_COLUMNS meta data on questions
|
||||
- one for each students
|
||||
|
||||
"""
|
||||
students = extract_students(df, no_student_columns)
|
||||
if student not in students:
|
||||
raise KeyError("This student is not in the table")
|
||||
st_df = df[list(no_student_columns) + [student]]
|
||||
st_df = st_df.rename(columns={student: COLUMNS["score"]}).dropna(
|
||||
subset=[COLUMNS["score"]]
|
||||
)
|
||||
if postprocessing:
|
||||
return postprocess(st_df)
|
||||
return st_df
|
||||
|
||||
|
||||
def postprocess(df):
|
||||
""" Postprocessing score dataframe
|
||||
|
||||
- Replace na with an empty string
|
||||
- Replace "NOANSWER" with -1
|
||||
- Turn commas number to dot numbers
|
||||
"""
|
||||
|
||||
df[COLUMNS["question"]].fillna("", inplace=True)
|
||||
df[COLUMNS["exercise"]].fillna("", inplace=True)
|
||||
df[COLUMNS["comment"]].fillna("", inplace=True)
|
||||
df[COLUMNS["competence"]].fillna("", inplace=True)
|
||||
|
||||
df[COLUMNS["score"]] = pd.to_numeric(
|
||||
df[COLUMNS["score"]]
|
||||
.replace(VALIDSCORE["NOANSWER"], -1)
|
||||
.apply(lambda x: try_replace(x, ",", "."))
|
||||
)
|
||||
df[COLUMNS["score_rate"]] = pd.to_numeric(
|
||||
df[COLUMNS["score_rate"]].apply(lambda x: try_replace(x, ",", ".")),
|
||||
errors="coerce",
|
||||
)
|
||||
|
||||
return df
|
||||
|
||||
|
||||
# -----------------------------
|
||||
# Reglages pour 'vim'
|
||||
# vim:set autoindent expandtab tabstop=4 shiftwidth=4:
|
||||
# cursor: 16 del
|
|
@ -0,0 +1,20 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
import dash
|
||||
import flask
|
||||
|
||||
from .layout.layout import layout
|
||||
|
||||
server = flask.Flask(__name__)
|
||||
app = dash.Dash(
|
||||
__name__,
|
||||
server=server,
|
||||
suppress_callback_exceptions=True,
|
||||
meta_tags=[{"name": "viewport", "content": "width=device-width, initial-scale=1"}],
|
||||
)
|
||||
|
||||
|
||||
app.layout = layout
|
||||
server = app.server
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
|
||||
def highlight_scores(highlight_columns, score_color):
|
||||
""" Cells style in a datatable for scores
|
||||
|
||||
:param highlight_columns: columns to highlight
|
||||
:param value_color: dictionnary {"score": "color"}
|
||||
|
||||
"""
|
||||
hight = []
|
||||
for v, color in score_color.items():
|
||||
if v:
|
||||
hight += [
|
||||
{
|
||||
"if": {"filter_query": "{{{}}} = {}".format(col, v), "column_id": col},
|
||||
"backgroundColor": color,
|
||||
"color": "white",
|
||||
}
|
||||
for col in highlight_columns
|
||||
]
|
||||
return hight
|
|
@ -0,0 +1,8 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from .app import app, server
|
||||
from .routes import render_page_content
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run_server(debug=True)
|
|
@ -0,0 +1,9 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
import dash_html_components as html
|
||||
import dash_core_components as dcc
|
||||
|
||||
content = html.Div(id="page-content")
|
||||
|
||||
layout = html.Div([dcc.Location(id="url"), content])
|
|
@ -0,0 +1,112 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
import dash_html_components as html
|
||||
import dash_core_components as dcc
|
||||
from .models import get_tribes, get_exams
|
||||
from .callbacks import *
|
||||
|
||||
layout = html.Div(
|
||||
children=[
|
||||
html.Header(
|
||||
children=[
|
||||
html.H1("Analyse des notes"),
|
||||
html.P("Dernière sauvegarde", id="lastsave"),
|
||||
],
|
||||
),
|
||||
html.Main(
|
||||
children=[
|
||||
html.Section(
|
||||
children=[
|
||||
html.Div(
|
||||
children=[
|
||||
"Classe: ",
|
||||
dcc.Dropdown(
|
||||
id="tribe",
|
||||
options=[
|
||||
{"label": t["name"], "value": t["name"]}
|
||||
for t in get_tribes().values()
|
||||
],
|
||||
value=next(iter(get_tribes().values()))["name"],
|
||||
),
|
||||
],
|
||||
),
|
||||
html.Div(
|
||||
children=[
|
||||
"Evaluation: ",
|
||||
dcc.Dropdown(id="exam_select"),
|
||||
],
|
||||
),
|
||||
],
|
||||
id="selects",
|
||||
),
|
||||
html.Section(
|
||||
children=[
|
||||
html.Div(
|
||||
children=[
|
||||
dash_table.DataTable(
|
||||
id="final_score_table",
|
||||
columns=[
|
||||
{"name": "Étudiant", "id": "student_name"},
|
||||
{"name": "Note", "id": "mark"},
|
||||
{"name": "Barème", "id": "score_rate"},
|
||||
],
|
||||
)
|
||||
],
|
||||
id="final_score_table_container",
|
||||
),
|
||||
html.Div(
|
||||
children=[
|
||||
dash_table.DataTable(
|
||||
id="score_statistics_table",
|
||||
columns=[],
|
||||
)
|
||||
],
|
||||
id="score_statistics_table_container",
|
||||
),
|
||||
html.Div(
|
||||
children=[
|
||||
dcc.Graph(
|
||||
id="fig_exam_histo",
|
||||
config={"displayModeBar": False},
|
||||
)
|
||||
],
|
||||
id="fig_exam_histo_container",
|
||||
),
|
||||
html.Div(
|
||||
children=[
|
||||
dcc.Graph(
|
||||
id="fig_questions_bar",
|
||||
config={"displayModeBar": False},
|
||||
)
|
||||
],
|
||||
id="fig_questions_bar_container",
|
||||
),
|
||||
],
|
||||
id="analysis",
|
||||
),
|
||||
html.Section(
|
||||
children=[
|
||||
dash_table.DataTable(
|
||||
id="scores_table",
|
||||
columns=[],
|
||||
style_data_conditional=[],
|
||||
fixed_columns={},
|
||||
editable=True,
|
||||
style_table={"minWidth": "100%"},
|
||||
style_cell={
|
||||
"minWidth": "100px",
|
||||
"width": "100px",
|
||||
"maxWidth": "100px",
|
||||
"overflow": "hidden",
|
||||
"textOverflow": "ellipsis",
|
||||
},
|
||||
)
|
||||
],
|
||||
id="edit",
|
||||
),
|
||||
],
|
||||
),
|
||||
dcc.Store(id="scores"),
|
||||
],
|
||||
)
|
|
@ -0,0 +1,216 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from dash.dependencies import Input, Output, State
|
||||
from dash.exceptions import PreventUpdate
|
||||
import plotly.graph_objects as go
|
||||
import dash_table
|
||||
import json
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
|
||||
from recopytex.dashboard.app import app
|
||||
from recopytex.dashboard.common.formating import highlight_scores
|
||||
|
||||
from .models import (
|
||||
get_tribes,
|
||||
get_exams,
|
||||
get_unstack_scores,
|
||||
get_students_from_exam,
|
||||
get_score_colors,
|
||||
get_level_color_bar,
|
||||
score_to_final_mark,
|
||||
stack_scores,
|
||||
pivot_score_on,
|
||||
)
|
||||
|
||||
|
||||
@app.callback(
|
||||
[
|
||||
Output("exam_select", "options"),
|
||||
Output("exam_select", "value"),
|
||||
],
|
||||
[Input("tribe", "value")],
|
||||
)
|
||||
def update_exams_choices(tribe):
|
||||
if not tribe:
|
||||
raise PreventUpdate
|
||||
exams = get_exams(tribe)
|
||||
exams.reset_index(inplace=True)
|
||||
if not exams.empty:
|
||||
return [
|
||||
{"label": e["name"], "value": e.to_json()} for i, e in exams.iterrows()
|
||||
], exams.loc[0].to_json()
|
||||
return [], None
|
||||
|
||||
|
||||
@app.callback(
|
||||
[
|
||||
Output("scores_table", "columns"),
|
||||
Output("scores_table", "data"),
|
||||
Output("scores_table", "style_data_conditional"),
|
||||
Output("scores_table", "fixed_columns"),
|
||||
],
|
||||
[
|
||||
Input("exam_select", "value"),
|
||||
],
|
||||
)
|
||||
def update_scores_store(exam):
|
||||
if not exam:
|
||||
return [[], [], [], {}]
|
||||
exam = pd.DataFrame.from_dict([json.loads(exam)])
|
||||
scores = get_unstack_scores(exam)
|
||||
fixed_columns = [
|
||||
"exercise",
|
||||
"question",
|
||||
"competence",
|
||||
"theme",
|
||||
"comment",
|
||||
"score_rate",
|
||||
"is_leveled",
|
||||
]
|
||||
|
||||
students = list(get_students_from_exam(exam))
|
||||
columns = fixed_columns + students
|
||||
|
||||
score_color = get_score_colors()
|
||||
|
||||
return [
|
||||
[{"id": c, "name": c} for c in columns],
|
||||
scores.to_dict("records"),
|
||||
highlight_scores(students, score_color),
|
||||
{"headers": True, "data": len(fixed_columns)},
|
||||
]
|
||||
|
||||
|
||||
@app.callback(
|
||||
[
|
||||
Output("final_score_table", "data"),
|
||||
],
|
||||
[
|
||||
Input("scores_table", "data"),
|
||||
],
|
||||
)
|
||||
def update_finale_score_table(scores):
|
||||
scores_df = pd.DataFrame.from_records(scores)
|
||||
stacked_scores = stack_scores(scores_df)
|
||||
return score_to_final_mark(stacked_scores)
|
||||
|
||||
|
||||
@app.callback(
|
||||
[
|
||||
Output("score_statistics_table", "columns"),
|
||||
Output("score_statistics_table", "data"),
|
||||
],
|
||||
[
|
||||
Input("final_score_table", "data"),
|
||||
],
|
||||
)
|
||||
def update_statictics_table(finale_score):
|
||||
df = pd.DataFrame.from_records(finale_score)
|
||||
statistics = df["mark"].describe().to_frame().T
|
||||
return [
|
||||
[{"id": c, "name": c} for c in statistics.columns],
|
||||
statistics.to_dict("records"),
|
||||
]
|
||||
|
||||
|
||||
@app.callback(
|
||||
[
|
||||
Output("fig_exam_histo", "figure"),
|
||||
],
|
||||
[
|
||||
Input("final_score_table", "data"),
|
||||
],
|
||||
)
|
||||
def update_exam_histo(finale_scores):
|
||||
scores = pd.DataFrame.from_records(finale_scores)
|
||||
|
||||
if scores.empty:
|
||||
return [go.Figure(data=[go.Scatter(x=[], y=[])])]
|
||||
|
||||
ranges = np.linspace(
|
||||
-0.5,
|
||||
scores["score_rate"].max(),
|
||||
int(scores["score_rate"].max() * 2 + 2),
|
||||
)
|
||||
|
||||
bins = pd.cut(scores["mark"], ranges)
|
||||
scores["Bin"] = bins
|
||||
grouped = (
|
||||
scores.reset_index()
|
||||
.groupby("Bin")
|
||||
.agg({"score_rate": "count", "student_name": lambda x: "\n".join(x)})
|
||||
)
|
||||
grouped.index = grouped.index.map(lambda i: i.right)
|
||||
fig = go.Figure()
|
||||
fig.add_bar(
|
||||
x=grouped.index,
|
||||
y=grouped["score_rate"],
|
||||
text=grouped["student_name"],
|
||||
textposition="auto",
|
||||
hovertemplate="",
|
||||
marker_color="#4E89DE",
|
||||
)
|
||||
fig.update_layout(
|
||||
height=300,
|
||||
margin=dict(l=5, r=5, b=5, t=5),
|
||||
)
|
||||
return [fig]
|
||||
|
||||
|
||||
@app.callback(
|
||||
[
|
||||
Output("fig_questions_bar", "figure"),
|
||||
],
|
||||
[
|
||||
Input("scores_table", "data"),
|
||||
],
|
||||
)
|
||||
def update_questions_bar(finale_scores):
|
||||
scores = pd.DataFrame.from_records(finale_scores)
|
||||
scores = stack_scores(scores)
|
||||
|
||||
if scores.empty:
|
||||
return [go.Figure(data=[go.Scatter(x=[], y=[])])]
|
||||
|
||||
pt = pivot_score_on(scores, ["exercise", "question", "comment"], "score")
|
||||
|
||||
# separation between exercises
|
||||
for i in {i for i in pt.index.get_level_values(0)}:
|
||||
pt.loc[(str(i), "", ""), :] = ""
|
||||
pt.sort_index(inplace=True)
|
||||
|
||||
# Bar label
|
||||
index = (
|
||||
pt.index.get_level_values(0).map(str)
|
||||
+ ":"
|
||||
+ pt.index.get_level_values(1).map(str)
|
||||
+ " "
|
||||
+ pt.index.get_level_values(2).map(str)
|
||||
)
|
||||
|
||||
fig = go.Figure()
|
||||
|
||||
bars = get_level_color_bar()
|
||||
|
||||
for b in bars:
|
||||
try:
|
||||
fig.add_bar(
|
||||
x=index, y=pt[b["score"]], name=b["name"], marker_color=b["color"]
|
||||
)
|
||||
except KeyError:
|
||||
pass
|
||||
fig.update_layout(barmode="relative")
|
||||
fig.update_layout(
|
||||
height=500,
|
||||
margin=dict(l=5, r=5, b=5, t=5),
|
||||
legend=dict(
|
||||
orientation="h",
|
||||
yanchor="bottom",
|
||||
y=1.02,
|
||||
xanchor="right",
|
||||
x=1
|
||||
)
|
||||
)
|
||||
return [fig]
|
|
@ -0,0 +1,128 @@
|
|||
#!/use/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from recopytex.database.filesystem.loader import CSVLoader
|
||||
from recopytex.datalib.dataframe import column_values_to_column
|
||||
import recopytex.datalib.on_score_column as on_column
|
||||
import pandas as pd
|
||||
|
||||
LOADER = CSVLoader("./test_confia.ml")
|
||||
SCORES_CONFIG = LOADER.get_config()["scores"]
|
||||
|
||||
|
||||
def unstack_scores(scores):
|
||||
"""Put student_name values to columns
|
||||
|
||||
:param scores: Score dataframe with one line per score
|
||||
:returns: Scrore dataframe with student_name in columns
|
||||
|
||||
"""
|
||||
kept_columns = [col for col in LOADER.score_columns if col != "score"]
|
||||
return column_values_to_column("student_name", "score", kept_columns, scores)
|
||||
|
||||
|
||||
def stack_scores(scores):
|
||||
"""Student columns are melt to rows with student_name column
|
||||
|
||||
:param scores: Score dataframe with student_name in columns
|
||||
:returns: Scrore dataframe with one line per score
|
||||
|
||||
"""
|
||||
kept_columns = [
|
||||
c for c in LOADER.score_columns if c not in ["score", "student_name"]
|
||||
]
|
||||
student_names = [c for c in scores.columns if c not in kept_columns]
|
||||
return pd.melt(
|
||||
scores,
|
||||
id_vars=kept_columns,
|
||||
value_vars=student_names,
|
||||
var_name="student_name",
|
||||
value_name="score",
|
||||
)
|
||||
|
||||
|
||||
def get_tribes():
|
||||
return LOADER.get_tribes()
|
||||
|
||||
|
||||
def get_exams(tribe):
|
||||
return LOADER.get_exams([tribe])
|
||||
|
||||
|
||||
def get_record_scores(exam):
|
||||
return LOADER.get_exam_scores(exam)
|
||||
|
||||
|
||||
def get_unstack_scores(exam):
|
||||
flat_scores = LOADER.get_exam_scores(exam)
|
||||
return unstack_scores(flat_scores)
|
||||
|
||||
|
||||
def get_students_from_exam(exam):
|
||||
flat_scores = LOADER.get_exam_scores(exam)
|
||||
return flat_scores["student_name"].unique()
|
||||
|
||||
|
||||
def get_score_colors():
|
||||
score_color = {}
|
||||
for key, score in SCORES_CONFIG.items():
|
||||
score_color[score["value"]] = score["color"]
|
||||
return score_color
|
||||
|
||||
|
||||
def get_level_color_bar():
|
||||
return [
|
||||
{"score": str(s["value"]), "name": s["comment"], "color": s["color"]}
|
||||
for s in SCORES_CONFIG.values()
|
||||
]
|
||||
|
||||
|
||||
is_none_score = lambda x: on_column.is_none_score(x, SCORES_CONFIG)
|
||||
format_score = lambda x: on_column.format_score(x, SCORES_CONFIG)
|
||||
score_to_numeric_score = lambda x: on_column.score_to_numeric_score(x, SCORES_CONFIG)
|
||||
score_to_mark = lambda x: on_column.score_to_mark(
|
||||
x, max([v["value"] for v in SCORES_CONFIG.values() if isinstance(v["value"], int)])
|
||||
)
|
||||
|
||||
|
||||
def filter_clean_score(scores):
|
||||
filtered_scores = scores[~scores.apply(is_none_score, axis=1)]
|
||||
filtered_scores = filtered_scores.assign(
|
||||
score=filtered_scores.apply(format_score, axis=1)
|
||||
)
|
||||
return filtered_scores
|
||||
|
||||
|
||||
def score_to_final_mark(scores):
|
||||
""" Compute marks then reduce to final mark per student """
|
||||
|
||||
filtered_scores = filter_clean_score(scores)
|
||||
filtered_scores = filtered_scores.assign(
|
||||
score=filtered_scores.apply(score_to_numeric_score, axis=1)
|
||||
)
|
||||
filtered_scores = filtered_scores.assign(
|
||||
mark=filtered_scores.apply(score_to_mark, axis=1)
|
||||
)
|
||||
final_score = filtered_scores.groupby(["student_name"])[
|
||||
["mark", "score_rate"]
|
||||
].sum()
|
||||
return [final_score.reset_index().to_dict("records")]
|
||||
|
||||
|
||||
def pivot_score_on(scores, index, columns, aggfunc="size"):
|
||||
"""Pivot scores on index, columns with aggfunc
|
||||
|
||||
It assumes thant scores are levels
|
||||
|
||||
"""
|
||||
filtered_scores = filter_clean_score(scores)
|
||||
filtered_scores["score"] = filtered_scores["score"].astype(str)
|
||||
pt = pd.pivot_table(
|
||||
filtered_scores,
|
||||
index=index,
|
||||
columns=columns,
|
||||
aggfunc=aggfunc,
|
||||
fill_value=0,
|
||||
)
|
||||
return pt
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
import dash_html_components as html
|
||||
from recopytex.database.filesystem.loader import CSVLoader
|
||||
from .models import get_tribes, get_exams, get_students
|
||||
|
||||
loader = CSVLoader("./test_config.yml")
|
||||
|
||||
|
||||
def listing(elements, formating=lambda x: x):
|
||||
|
||||
return html.Ul(
|
||||
children=[html.Li(children=formating(element)) for element in elements]
|
||||
)
|
||||
|
||||
|
||||
def format_tribe(tribe):
|
||||
children = [html.H3(tribe["name"])]
|
||||
exams = loader.get_exams([tribe["name"]])
|
||||
if exams.empty:
|
||||
children.append(html.P("Pas d'évaluation"))
|
||||
else:
|
||||
exams_html = listing([exam for id, exam in exams.iterrows()], format_exam)
|
||||
children.append(exams_html)
|
||||
return children
|
||||
|
||||
|
||||
def format_exam(exam):
|
||||
children = [html.P(exam["name"])]
|
||||
return children
|
||||
|
||||
|
||||
layout = html.Div(
|
||||
children=[
|
||||
html.H1("Recopytex"),
|
||||
html.H2("Tribes"),
|
||||
html.Div(
|
||||
children=[listing(loader.get_tribes().values(), format_tribe)],
|
||||
id="tribes",
|
||||
),
|
||||
html.H2("Config"),
|
||||
html.Div(
|
||||
children=[
|
||||
html.P(str(loader.get_config())),
|
||||
],
|
||||
id="config",
|
||||
),
|
||||
]
|
||||
)
|
|
@ -0,0 +1,6 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from dash.dependencies import Input, Output
|
||||
from recopytex.dashboard.app import app
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
|
||||
def get_tribes(loader):
|
||||
return loader.get_tribes()
|
||||
|
||||
|
||||
def get_exams(loader, tribe):
|
||||
return loader.get_exams([tribe])
|
||||
|
||||
|
||||
def get_students(loader, tribe):
|
||||
return loader.get_students([tribe])
|
|
@ -0,0 +1,27 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from dash.dependencies import Input, Output
|
||||
|
||||
from .app import app
|
||||
from .pages.home import app as home
|
||||
from .pages.exams_scores import app as exams_scores
|
||||
import dash_html_components as html
|
||||
|
||||
|
||||
@app.callback(Output("page-content", "children"), [Input("url", "pathname")])
|
||||
def render_page_content(pathname):
|
||||
if pathname == "/":
|
||||
return home.layout
|
||||
elif pathname == "/exams/scores/":
|
||||
return exams_scores.layout
|
||||
# elif pathname == iris_page_location:
|
||||
# return iris.layout
|
||||
# # If the user tries to reach a different page, return a 404 message
|
||||
return html.Div(
|
||||
[
|
||||
html.H1("404: Not found", className="text-danger"),
|
||||
html.Hr(),
|
||||
html.P(f"The pathname {pathname} was not recognised..."),
|
||||
]
|
||||
)
|
|
@ -0,0 +1,88 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
import yaml
|
||||
|
||||
"""
|
||||
|
||||
Adapter to pull data from the filesystem
|
||||
|
||||
# Loader
|
||||
|
||||
# Writer
|
||||
"""
|
||||
|
||||
|
||||
class Loader(ABC):
|
||||
|
||||
"""Load data from source"""
|
||||
|
||||
CONFIG = {}
|
||||
|
||||
def __init__(self, configfile=""):
|
||||
"""Init loader
|
||||
|
||||
:param configfile: yaml file with informations on data source
|
||||
"""
|
||||
self._config = self.CONFIG
|
||||
if configfile.endswith(".yml"):
|
||||
with open(configfile, "r") as config:
|
||||
self._config.update(yaml.load(config, Loader=yaml.FullLoader))
|
||||
|
||||
def get_config(self):
|
||||
""" Get config"""
|
||||
return self._config
|
||||
|
||||
@abstractmethod
|
||||
def get_tribes(self):
|
||||
""" Get tribes list """
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_exams(self, tribes=[]):
|
||||
"""Get exams list
|
||||
|
||||
:param tribes: get only exams for those tribes
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_students(self, tribes=[]):
|
||||
"""Get student list
|
||||
|
||||
:param filters: list of filters
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_exam_questions(self, exams=[]):
|
||||
"""Get questions for the exam
|
||||
|
||||
:param exams: questions for those exams only
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_questions_scores(self, questions=[]):
|
||||
"""Get scores of those questions
|
||||
|
||||
:param questions: score for those questions
|
||||
"""
|
||||
pass
|
||||
|
||||
# @abstractmethod
|
||||
# def get_student_scores(self, student):
|
||||
# """Get scores of the student
|
||||
|
||||
# :param student:
|
||||
# """
|
||||
# pass
|
||||
|
||||
|
||||
class Writer(ABC):
|
||||
|
||||
""" Write datas to the source """
|
||||
|
||||
def __init__(self):
|
||||
pass
|
|
@ -0,0 +1,15 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
"""
|
||||
Store data using filesystem for organisation, csv for scores
|
||||
|
||||
## Organisation
|
||||
|
||||
- tribe1.csv # list of students for the tribe
|
||||
- tribe1/
|
||||
- exam1.csv # questions and scores for exam1
|
||||
- exam1.yml # Extra information about exam1
|
||||
- exam2.csv # questions and scores for exam2
|
||||
"""
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
---
|
||||
source: ./ # basepath where to start
|
||||
|
||||
competences: # Competences
|
||||
Chercher:
|
||||
name: Chercher
|
||||
abrv: Cher
|
||||
Représenter:
|
||||
name: Représenter
|
||||
abrv: Rep
|
||||
Modéliser:
|
||||
name: Modéliser
|
||||
abrv: Mod
|
||||
Raisonner:
|
||||
name: Raisonner
|
||||
abrv: Rai
|
||||
Calculer:
|
||||
name: Calculer
|
||||
abrv: Cal
|
||||
Communiquer:
|
||||
name: Communiquer
|
||||
abrv: Com
|
||||
|
||||
scores: #
|
||||
BAD: # Everything is bad
|
||||
value: 0
|
||||
numeric_value: 0
|
||||
color: "#E7472B"
|
||||
comment: Faux
|
||||
FEW: # Few good things
|
||||
value: 1
|
||||
numeric_value: 1
|
||||
color: "#FF712B"
|
||||
comment: Peu juste
|
||||
NEARLY: # Nearly good but things are missing
|
||||
value: 2
|
||||
numeric_value: 2
|
||||
color: "#F2EC4C"
|
||||
comment: Presque juste
|
||||
GOOD: # Everything is good
|
||||
value: 3
|
||||
numeric_value: 3
|
||||
color: "#68D42F"
|
||||
comment: Juste
|
||||
NOTFILLED: # The item is not scored yet
|
||||
value: ""
|
||||
numeric_value: None
|
||||
color: white
|
||||
comment: En attente
|
||||
NOANSWER: # Student gives no answer (count as 0)
|
||||
value: "."
|
||||
numeric_value: 0
|
||||
color: black
|
||||
comment: Pas de réponse
|
||||
ABS: # Student has absent (this score won't be impact the final mark)
|
||||
value: a
|
||||
numeric_value: None
|
||||
color: lightgray
|
||||
comment: Non noté
|
||||
|
||||
csv_fields: # dataframe_field: csv_field
|
||||
term: Trimestre
|
||||
exam: Nom
|
||||
date: Date
|
||||
exercise: Exercice
|
||||
question: Question
|
||||
competence: Competence
|
||||
theme: Domaine
|
||||
comment: Commentaire
|
||||
score_rate: Bareme
|
||||
is_leveled: Est_nivele
|
||||
|
||||
id_templates:
|
||||
exam: "{name}_{tribe}"
|
||||
question: "{exam_id}_{exercise}_{question}_{comment}"
|
|
@ -0,0 +1,52 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
import pandas as pd
|
||||
from pathlib import Path
|
||||
from unidecode import unidecode
|
||||
|
||||
|
||||
__all__ = ["list_csvs", "extract_fields"]
|
||||
|
||||
|
||||
def list_csvs(path):
|
||||
"""list csv files in path
|
||||
|
||||
:example:
|
||||
>>> list_csvs("./example/Tribe1/")
|
||||
[PosixPath('example/Tribe1/210112_DS.csv'), PosixPath('example/Tribe1/210122_DS6.csv')]
|
||||
>>> list_csvs("./example/Tribe1")
|
||||
[PosixPath('example/Tribe1/210112_DS.csv'), PosixPath('example/Tribe1/210122_DS6.csv')]
|
||||
"""
|
||||
return list(Path(path).glob("*.csv"))
|
||||
|
||||
|
||||
def extract_fields(csv_filename, fields=[], remove_duplicates=True):
|
||||
"""Extract fields in csv
|
||||
|
||||
:param csv_filename: csv filename (with header)
|
||||
:param fields: list of fields to extract (all fields if empty list - default)
|
||||
:param remove_duplicates: keep uniques rows (default True)
|
||||
|
||||
:example:
|
||||
>>> extract_fields("./example/Tribe1/210122_DS6.csv", ["Trimestre", "Nom", "Date"])
|
||||
Trimestre Nom Date
|
||||
0 1 DS6 22/01/2021
|
||||
"""
|
||||
df = pd.read_csv(csv_filename)
|
||||
if fields:
|
||||
df = df[fields]
|
||||
if remove_duplicates:
|
||||
return df.drop_duplicates()
|
||||
return df
|
||||
|
||||
|
||||
def build_id(template, element):
|
||||
"""Build an id from template to the element
|
||||
|
||||
:example:
|
||||
>>> element = {"name": "pléà", "place": "here", "foo":"bar"}
|
||||
>>> build_id("{name} {place}", element)
|
||||
'plea_here'
|
||||
"""
|
||||
return unidecode(template.format(**element)).replace(" ", "_")
|
|
@ -0,0 +1,298 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
import yaml
|
||||
import os
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
import pandas as pd
|
||||
from .. import Loader
|
||||
from .lib import list_csvs, extract_fields, build_id
|
||||
|
||||
|
||||
DEFAULT_CONFIG_FILE = os.path.join(os.path.dirname(__file__), "default_config.yml")
|
||||
with open(DEFAULT_CONFIG_FILE, "r") as config:
|
||||
DEFAULT_CONFIG = yaml.load(config, Loader=yaml.FullLoader)
|
||||
|
||||
|
||||
def maybe_dataframe(datas):
|
||||
try:
|
||||
return [e[1] for e in datas.iterrows()]
|
||||
except AttributeError:
|
||||
return datas
|
||||
|
||||
|
||||
class CSVLoader(Loader):
|
||||
|
||||
"""Loader when scores and metadatas are stored in csv files
|
||||
|
||||
:config:
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader()
|
||||
>>> loader.get_config()
|
||||
{'source': './', 'competences': {'Chercher': {'name': 'Chercher', 'abrv': 'Cher'}, 'Représenter': {'name': 'Représenter', 'abrv': 'Rep'}, 'Modéliser': {'name': 'Modéliser', 'abrv': 'Mod'}, 'Raisonner': {'name': 'Raisonner', 'abrv': 'Rai'}, 'Calculer': {'name': 'Calculer', 'abrv': 'Cal'}, 'Communiquer': {'name': 'Communiquer', 'abrv': 'Com'}}, 'scores': {'BAD': {'value': 0, 'numeric_value': 0, 'color': '#E7472B', 'comment': 'Faux'}, 'FEW': {'value': 1, 'numeric_value': 1, 'color': '#FF712B', 'comment': 'Peu juste'}, 'NEARLY': {'value': 2, 'numeric_value': 2, 'color': '#F2EC4C', 'comment': 'Presque juste'}, 'GOOD': {'value': 3, 'numeric_value': 3, 'color': '#68D42F', 'comment': 'Juste'}, 'NOTFILLED': {'value': '', 'numeric_value': 'None', 'color': 'white', 'comment': 'En attente'}, 'NOANSWER': {'value': '.', 'numeric_value': 0, 'color': 'black', 'comment': 'Pas de réponse'}, 'ABS': {'value': 'a', 'numeric_value': 'None', 'color': 'lightgray', 'comment': 'Non noté'}}, 'csv_fields': {'term': 'Trimestre', 'exam': 'Nom', 'date': 'Date', 'exercise': 'Exercice', 'question': 'Question', 'competence': 'Competence', 'theme': 'Domaine', 'comment': 'Commentaire', 'score_rate': 'Bareme', 'is_leveled': 'Est_nivele'}, 'id_templates': {'exam': '{name}_{tribe}', 'question': '{exam_id}_{exercise}_{question}_{comment}'}}
|
||||
|
||||
>>> loader = CSVLoader("./test_config.yml")
|
||||
>>> loader.get_config()
|
||||
{'source': './example', 'competences': {'Chercher': {'name': 'Chercher', 'abrv': 'Cher'}, 'Représenter': {'name': 'Représenter', 'abrv': 'Rep'}, 'Modéliser': {'name': 'Modéliser', 'abrv': 'Mod'}, 'Raisonner': {'name': 'Raisonner', 'abrv': 'Rai'}, 'Calculer': {'name': 'Calculer', 'abrv': 'Cal'}, 'Communiquer': {'name': 'Communiquer', 'abrv': 'Com'}}, 'scores': {'BAD': {'value': 0, 'numeric_value': 0, 'color': '#E7472B', 'comment': 'Faux'}, 'FEW': {'value': 1, 'numeric_value': 1, 'color': '#FF712B', 'comment': 'Peu juste'}, 'NEARLY': {'value': 2, 'numeric_value': 2, 'color': '#F2EC4C', 'comment': 'Presque juste'}, 'GOOD': {'value': 3, 'numeric_value': 3, 'color': '#68D42F', 'comment': 'Juste'}, 'NOTFILLED': {'value': '', 'numeric_value': 'None', 'color': 'white', 'comment': 'En attente'}, 'NOANSWER': {'value': '.', 'numeric_value': 0, 'color': 'black', 'comment': 'Pas de réponse'}, 'ABS': {'value': 'a', 'numeric_value': 'None', 'color': 'lightgray', 'comment': 'Non noté'}}, 'csv_fields': {'term': 'Trimestre', 'exam': 'Nom', 'date': 'Date', 'exercise': 'Exercice', 'question': 'Question', 'competence': 'Competence', 'theme': 'Domaine', 'comment': 'Commentaire', 'score_rate': 'Bareme', 'is_leveled': 'Est_nivele'}, 'id_templates': {'exam': '{name}_{tribe}', 'question': '{exam_id}_{exercise}_{question}_{comment}'}, 'output': './output', 'templates': 'templates/', 'tribes': {'Tribe1': {'name': 'Tribe1', 'type': 'Type1', 'students': 'tribe1.csv'}, 'Tribe2': {'name': 'Tribe2', 'students': 'tribe2.csv'}}}
|
||||
"""
|
||||
|
||||
CONFIG = DEFAULT_CONFIG
|
||||
|
||||
def get_config(self):
|
||||
""" Get config """
|
||||
return self._config
|
||||
|
||||
@property
|
||||
def exam_columns(self):
|
||||
return pd.Index(["name", "date", "term", "origin", "tribe", "id"])
|
||||
|
||||
@property
|
||||
def question_columns(self):
|
||||
return pd.Index(
|
||||
[
|
||||
"exercise",
|
||||
"question",
|
||||
"competence",
|
||||
"theme",
|
||||
"comment",
|
||||
"score_rate",
|
||||
"is_leveled",
|
||||
"origin",
|
||||
"exam_id",
|
||||
"id",
|
||||
]
|
||||
)
|
||||
|
||||
@property
|
||||
def score_columns(self):
|
||||
return pd.Index(
|
||||
[
|
||||
"term",
|
||||
"exam",
|
||||
"date",
|
||||
"exercise",
|
||||
"question",
|
||||
"competence",
|
||||
"theme",
|
||||
"comment",
|
||||
"score_rate",
|
||||
"is_leveled",
|
||||
"origin",
|
||||
"exam_id",
|
||||
"question_id",
|
||||
"student_name",
|
||||
"score",
|
||||
]
|
||||
)
|
||||
|
||||
def rename_columns(self, dataframe):
|
||||
"""Rename dataframe column to match with `csv_fields`
|
||||
|
||||
:param dataframe: the dataframe
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader()
|
||||
>>>
|
||||
|
||||
"""
|
||||
return dataframe.rename(
|
||||
columns={v: k for k, v in self._config["csv_fields"].items()}
|
||||
)
|
||||
|
||||
def reverse_csv_field(self, keys):
|
||||
""" Reverse csv field from keys """
|
||||
return [self._config["csv_fields"][k] for k in keys]
|
||||
|
||||
def get_tribes(self, only_names=False):
|
||||
"""Get tribes list
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader("./test_config.yml")
|
||||
>>> loader.get_tribes()
|
||||
{'Tribe1': {'name': 'Tribe1', 'type': 'Type1', 'students': 'tribe1.csv'}, 'Tribe2': {'name': 'Tribe2', 'students': 'tribe2.csv'}}
|
||||
>>> loader.get_tribes(only_names=True)
|
||||
['Tribe1', 'Tribe2']
|
||||
"""
|
||||
if only_names:
|
||||
return list(self._config["tribes"].keys())
|
||||
return self._config["tribes"]
|
||||
|
||||
def get_exams(self, tribes=[]):
|
||||
"""Get exams list
|
||||
|
||||
:param tribes: get only exams for those tribes
|
||||
:return: dataframe of exams
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader("./test_config.yml")
|
||||
>>> exams = loader.get_exams(["Tribe1"])
|
||||
>>> all(exams.columns == loader.exam_columns)
|
||||
True
|
||||
>>> exams
|
||||
name date term origin tribe id
|
||||
0 DS 12/01/2021 1 example/Tribe1/210112_DS.csv Tribe1 DS_Tribe1
|
||||
0 DS6 22/01/2021 1 example/Tribe1/210122_DS6.csv Tribe1 DS6_Tribe1
|
||||
"""
|
||||
exams = []
|
||||
for tribe in tribes:
|
||||
tribe_path = Path(self._config["source"]) / tribe
|
||||
csvs = list_csvs(tribe_path)
|
||||
for csv in csvs:
|
||||
fields = self.reverse_csv_field(["exam", "date", "term"])
|
||||
exam = extract_fields(csv, fields)
|
||||
exam = self.rename_columns(exam)
|
||||
exam = exam.rename(columns={"exam": "name"})
|
||||
exam["origin"] = str(csv)
|
||||
exam["tribe"] = tribe
|
||||
exam["id"] = build_id(
|
||||
self._config["id_templates"]["exam"], exam.iloc[0]
|
||||
)
|
||||
exams.append(exam)
|
||||
if exams:
|
||||
return pd.concat(exams)
|
||||
return pd.DataFrame(columns=["name", "date", "term", "origin", "tribe", "id"])
|
||||
|
||||
def get_exam_questions(self, exams=[]):
|
||||
"""Get questions for exams stored in score_files
|
||||
|
||||
:param exams: list or dataframe of exams metadatas (need origin field to find the csv)
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader("./test_config.yml")
|
||||
>>> exams = loader.get_exams(["Tribe1"])
|
||||
>>> all(loader.get_exam_questions([exams.iloc[0]]).columns == loader.question_columns)
|
||||
True
|
||||
>>> questions = loader.get_exam_questions(exams)
|
||||
>>> questions.iloc[0]
|
||||
exercise Exercice 1
|
||||
question 1
|
||||
competence Calculer
|
||||
theme Plop
|
||||
comment Coucou
|
||||
score_rate 1.0
|
||||
is_leveled 1.0
|
||||
origin example/Tribe1/210112_DS.csv
|
||||
exam_id DS_Tribe1
|
||||
id DS_Tribe1_Exercice_1_1_Coucou
|
||||
Name: 0, dtype: object
|
||||
"""
|
||||
_exams = maybe_dataframe(exams)
|
||||
|
||||
questions = []
|
||||
for exam in _exams:
|
||||
fields = self.reverse_csv_field(
|
||||
[
|
||||
"exercise",
|
||||
"question",
|
||||
"competence",
|
||||
"theme",
|
||||
"comment",
|
||||
"score_rate",
|
||||
"is_leveled",
|
||||
]
|
||||
)
|
||||
question = extract_fields(exam["origin"], fields)
|
||||
question = self.rename_columns(question)
|
||||
question["origin"] = exam["origin"]
|
||||
question["exam_id"] = exam["id"]
|
||||
question["id"] = build_id(
|
||||
self._config["id_templates"]["question"], question.iloc[0]
|
||||
)
|
||||
questions.append(question)
|
||||
|
||||
return pd.concat(questions)
|
||||
|
||||
def get_questions_scores(self, questions=[]):
|
||||
"""Get scores of those questions
|
||||
|
||||
:param questions: list or dataframe of questions metadatas (need origin field to find the csv)
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader("./test_config.yml")
|
||||
>>> exams = loader.get_exams(["Tribe1"])
|
||||
>>> questions = loader.get_exam_questions(exams)
|
||||
>>> scores = loader.get_questions_scores(questions)
|
||||
>>> all(scores.columns == loader.score_columns)
|
||||
True
|
||||
>>> scores["student_name"].unique()
|
||||
array(['Star Tice', 'Umberto Dingate', 'Starlin Crangle',
|
||||
'Humbert Bourcq', 'Gabriella Handyside', 'Stewart Eaves',
|
||||
'Erick Going', 'Ase Praton', 'Rollins Planks', 'Dunstan Sarjant',
|
||||
'Stacy Guiton', 'Ange Stanes', 'Amabelle Elleton',
|
||||
'Darn Broomhall', 'Dyan Chatto', 'Keane Rennebach', 'Nari Paulton',
|
||||
'Brandy Wase', 'Jaclyn Firidolfi', 'Violette Lockney'],
|
||||
dtype=object)
|
||||
"""
|
||||
scores = []
|
||||
group_questions = questions.groupby("origin")
|
||||
for origin, questions_df in group_questions:
|
||||
scores_df = pd.read_csv(origin)
|
||||
scores_df = self.rename_columns(scores_df)
|
||||
student_names = [
|
||||
c
|
||||
for c in scores_df.columns
|
||||
if c not in self._config["csv_fields"].keys()
|
||||
]
|
||||
|
||||
common_columns = [c for c in questions_df.columns if c in scores_df.columns]
|
||||
scores_df = pd.merge(scores_df, questions_df, on=common_columns)
|
||||
|
||||
kept_columns = [c for c in scores_df if c not in student_names]
|
||||
scores_df = pd.melt(
|
||||
scores_df,
|
||||
id_vars=kept_columns,
|
||||
value_vars=student_names,
|
||||
var_name="student_name",
|
||||
value_name="score",
|
||||
)
|
||||
|
||||
scores_df = scores_df.rename(columns={"id": "question_id"})
|
||||
scores.append(scores_df)
|
||||
|
||||
return pd.concat(scores)
|
||||
|
||||
def get_exam_scores(self, exams=[]):
|
||||
"""Get scores for all question of the exam
|
||||
|
||||
:param exams: list or dataframe of exams metadatas (need origin field to find the csv)
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader("./test_config.yml")
|
||||
>>> exams = loader.get_exams(["Tribe1"])
|
||||
>>> scores = loader.get_exam_scores(exams)
|
||||
>>> scores.columns
|
||||
Index(['term', 'exam', 'date', 'exercise', 'question', 'competence', 'theme',
|
||||
'comment', 'score_rate', 'is_leveled', 'origin', 'exam_id',
|
||||
'question_id', 'student_name', 'score'],
|
||||
dtype='object')
|
||||
"""
|
||||
questions = self.get_exam_questions(exams)
|
||||
return self.get_questions_scores(questions)
|
||||
|
||||
def get_students(self, tribes=[]):
|
||||
"""Get student list
|
||||
|
||||
:param tribes: concerned tribes
|
||||
|
||||
:example:
|
||||
>>> loader = CSVLoader("./test_config.yml")
|
||||
>>> tribes = loader.get_tribes()
|
||||
>>> students = loader.get_students([tribes["Tribe1"]])
|
||||
>>> students.columns
|
||||
Index(['Nom', 'email', 'origin', 'tribe'], dtype='object')
|
||||
"""
|
||||
students = []
|
||||
for tribe in tribes:
|
||||
students_csv = Path(self._config["source"]) / tribe["students"]
|
||||
students_df = pd.read_csv(students_csv)
|
||||
students_df["origin"] = students_csv
|
||||
students_df["tribe"] = tribe["name"]
|
||||
students.append(students_df)
|
||||
|
||||
return pd.concat(students)
|
||||
|
||||
def get_student_scores(self, student=[]):
|
||||
"""Get all scores for students"""
|
||||
pass
|
|
@ -0,0 +1,7 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
"""
|
||||
|
||||
"""
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
|
||||
def column_values_to_column(pivot_column, value_column, kept_columns, df):
|
||||
"""Pivot_column's values go to column with value_column under it, keeping kept_columns
|
||||
|
||||
:param pivot_column: column name where value will become columns
|
||||
:param value_column: column name where value will be under pivot_column
|
||||
:param kept_columns: unchanged columns
|
||||
:param df: DataFrame to work with
|
||||
|
||||
:return: Stack dataframe
|
||||
|
||||
"""
|
||||
if pivot_column in kept_columns:
|
||||
pivot_columns = kept_columns
|
||||
else:
|
||||
pivot_columns = kept_columns + [pivot_column]
|
||||
|
||||
return df.set_index(pivot_columns).unstack(pivot_column)[value_column].reset_index()
|
|
@ -0,0 +1,257 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from math import ceil
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def is_none_score(x, score_config):
|
||||
"""Is a score correspond to a None numeric_value which
|
||||
|
||||
>>> import pandas as pd
|
||||
>>> d = {"Eleve":["E1"]*7,
|
||||
... "score_rate": [1]*7,
|
||||
... "is_leveled":[0]+[1]*6,
|
||||
... "score":[0.33, "", ".", "a", 1, 2, 3],
|
||||
... }
|
||||
>>> score_config = {
|
||||
... 'BAD': {'value': 0, 'numeric_value': 0},
|
||||
... 'FEW': {'value': 1, 'numeric_value': 1},
|
||||
... 'NEARLY': {'value': 2, 'numeric_value': 2},
|
||||
... 'GOOD': {'value': 3, 'numeric_value': 3},
|
||||
... 'NOTFILLED': {'value': '', 'numeric_value': 'None'},
|
||||
... 'NOANSWER': {'value': '.', 'numeric_value': 0},
|
||||
... 'ABS': {'value': 'a', 'numeric_value': 'None'}
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> df.apply(lambda x:is_none_score(x, score_config), axis=1)
|
||||
0 False
|
||||
1 True
|
||||
2 False
|
||||
3 True
|
||||
4 False
|
||||
5 False
|
||||
6 False
|
||||
dtype: bool
|
||||
|
||||
"""
|
||||
none_values = [
|
||||
v["value"]
|
||||
for v in score_config.values()
|
||||
if str(v["numeric_value"]).lower() == "none"
|
||||
]
|
||||
return x["score"] in none_values or pd.isnull(x["score"])
|
||||
|
||||
|
||||
def format_score(x, score_config):
|
||||
"""Make sure that score have the appropriate format
|
||||
|
||||
>>> import pandas as pd
|
||||
>>> d = {"Eleve":["E1"]*6,
|
||||
... "score_rate": [1]*6,
|
||||
... "is_leveled":[0]+[1]*5,
|
||||
... "score":[0.33, ".", "a", 1, 2, 3],
|
||||
... }
|
||||
>>> score_config = {
|
||||
... 'BAD': {'value': 0, 'numeric_value': 0},
|
||||
... 'FEW': {'value': 1, 'numeric_value': 1},
|
||||
... 'NEARLY': {'value': 2, 'numeric_value': 2},
|
||||
... 'GOOD': {'value': 3, 'numeric_value': 3},
|
||||
... 'NOTFILLED': {'value': '', 'numeric_value': 'None'},
|
||||
... 'NOANSWER': {'value': '.', 'numeric_value': 0},
|
||||
... 'ABS': {'value': 'a', 'numeric_value': 'None'}
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> df.apply(lambda x:format_score(x, score_config), axis=1)
|
||||
0 0.33
|
||||
1 .
|
||||
2 a
|
||||
3 1
|
||||
4 2
|
||||
5 3
|
||||
dtype: object
|
||||
>>> format_score({"score": "1.0", "is_leveled": 1}, score_config)
|
||||
1
|
||||
>>> format_score({"score": "3.0", "is_leveled": 1}, score_config)
|
||||
3
|
||||
>>> format_score({"score": 4, "is_leveled": 1}, score_config)
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
ValueError: 4 (<class 'int'>) can't be a score
|
||||
|
||||
"""
|
||||
if not x["is_leveled"]:
|
||||
return float(x["score"])
|
||||
|
||||
try:
|
||||
score = int(float(x["score"]))
|
||||
except ValueError:
|
||||
score = str(x["score"])
|
||||
|
||||
if score in [v["value"] for v in score_config.values()]:
|
||||
return score
|
||||
|
||||
raise ValueError(f"{x['score']} ({type(x['score'])}) can't be a score")
|
||||
|
||||
|
||||
def score_to_numeric_score(x, score_config):
|
||||
"""Convert a score to the corresponding numeric value
|
||||
|
||||
>>> import pandas as pd
|
||||
>>> d = {"Eleve":["E1"]*7,
|
||||
... "score_rate": [1]*7,
|
||||
... "is_leveled":[0]+[1]*6,
|
||||
... "score":[0.33, "", ".", "a", 1, 2, 3],
|
||||
... }
|
||||
>>> score_config = {
|
||||
... 'BAD': {'value': 0, 'numeric_value': 0},
|
||||
... 'FEW': {'value': 1, 'numeric_value': 1},
|
||||
... 'NEARLY': {'value': 2, 'numeric_value': 2},
|
||||
... 'GOOD': {'value': 3, 'numeric_value': 3},
|
||||
... 'NOTFILLED': {'value': '', 'numeric_value': 'None'},
|
||||
... 'NOANSWER': {'value': '.', 'numeric_value': 0},
|
||||
... 'ABS': {'value': 'a', 'numeric_value': 'None'}
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> df.apply(lambda x:score_to_numeric_score(x, score_config), axis=1)
|
||||
0 0.33
|
||||
1 None
|
||||
2 0
|
||||
3 None
|
||||
4 1
|
||||
5 2
|
||||
6 3
|
||||
dtype: object
|
||||
|
||||
"""
|
||||
if x["is_leveled"]:
|
||||
replacements = {v["value"]: v["numeric_value"] for v in score_config.values()}
|
||||
return replacements[x["score"]]
|
||||
|
||||
return x["score"]
|
||||
|
||||
|
||||
def score_to_mark(x, score_max, rounding=lambda x: round(x, 2)):
|
||||
"""Compute the mark from "score" which have to be filtered and in numeric form
|
||||
|
||||
if the item is leveled then the score is multiply by the score_rate
|
||||
otherwise it copies the score
|
||||
|
||||
:param x: dictionnary with "is_leveled", "score" (need to be number) and "score_rate" keys
|
||||
:param score_max:
|
||||
:param rounding: rounding mark function
|
||||
:return: the mark
|
||||
|
||||
>>> import pandas as pd
|
||||
>>> d = {"Eleve":["E1"]*7,
|
||||
... "score_rate": [1]*7,
|
||||
... "is_leveled":[0]+[1]*6,
|
||||
... "score":[0.33, "", ".", "a", 1, 2, 3],
|
||||
... }
|
||||
>>> score_config = {
|
||||
... 'BAD': {'value': 0, 'numeric_value': 0},
|
||||
... 'FEW': {'value': 1, 'numeric_value': 1},
|
||||
... 'NEARLY': {'value': 2, 'numeric_value': 2},
|
||||
... 'GOOD': {'value': 3, 'numeric_value': 3},
|
||||
... 'NOTFILLED': {'value': '', 'numeric_value': 'None'},
|
||||
... 'NOANSWER': {'value': '.', 'numeric_value': 0},
|
||||
... 'ABS': {'value': 'a', 'numeric_value': 'None'}
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> df = df[~df.apply(lambda x:is_none_score(x, score_config), axis=1)]
|
||||
>>> df["score"] = df.apply(lambda x:score_to_numeric_score(x, score_config), axis=1)
|
||||
>>> df.apply(lambda x:score_to_mark(x, 3), axis=1)
|
||||
0 0.33
|
||||
2 0.00
|
||||
4 0.33
|
||||
5 0.67
|
||||
6 1.00
|
||||
dtype: float64
|
||||
>>> from .on_value import round_half_point
|
||||
>>> df.apply(lambda x:score_to_mark(x, 3, round_half_point), axis=1)
|
||||
0 0.5
|
||||
2 0.0
|
||||
4 0.5
|
||||
5 0.5
|
||||
6 1.0
|
||||
dtype: float64
|
||||
"""
|
||||
if x["is_leveled"]:
|
||||
if x["score"] not in list(range(score_max + 1)):
|
||||
raise ValueError(f"The evaluation is out of range: {x['score']} at {x}")
|
||||
return rounding(x["score"] * x["score_rate"] / score_max)
|
||||
|
||||
return rounding(x["score"])
|
||||
|
||||
|
||||
def score_to_level(x, level_max=3):
|
||||
"""Compute the level (".",0,1,2,3).
|
||||
|
||||
:param x: dictionnary with "is_leveled", "score" and "score_rate" keys
|
||||
:return: the level
|
||||
|
||||
>>> import pandas as pd
|
||||
>>> d = {"Eleve":["E1"]*6 + ["E2"]*6,
|
||||
... "score_rate":[1]*2+[2]*2+[2]*2 + [1]*2+[2]*2+[2]*2,
|
||||
... "is_leveled":[0]*4+[1]*2 + [0]*4+[1]*2,
|
||||
... "score":[1, 0.33, 0, 1.5, 1, 3, 0.666, 1, 1.5, 1, 2, 3],
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> df
|
||||
Eleve score_rate is_leveled score
|
||||
0 E1 1 0 1.000
|
||||
1 E1 1 0 0.330
|
||||
2 E1 2 0 0.000
|
||||
3 E1 2 0 1.500
|
||||
4 E1 2 1 1.000
|
||||
5 E1 2 1 3.000
|
||||
6 E2 1 0 0.666
|
||||
7 E2 1 0 1.000
|
||||
8 E2 2 0 1.500
|
||||
9 E2 2 0 1.000
|
||||
10 E2 2 1 2.000
|
||||
11 E2 2 1 3.000
|
||||
>>> df.apply(score_to_level, axis=1)
|
||||
0 3
|
||||
1 1
|
||||
2 0
|
||||
3 3
|
||||
4 1
|
||||
5 3
|
||||
6 2
|
||||
7 3
|
||||
8 3
|
||||
9 2
|
||||
10 2
|
||||
11 3
|
||||
dtype: int64
|
||||
>>> df.apply(lambda x: score_to_level(x, 5), axis=1)
|
||||
0 5
|
||||
1 2
|
||||
2 0
|
||||
3 4
|
||||
4 1
|
||||
5 3
|
||||
6 4
|
||||
7 5
|
||||
8 4
|
||||
9 3
|
||||
10 2
|
||||
11 3
|
||||
dtype: int64
|
||||
"""
|
||||
if x["is_leveled"]:
|
||||
return int(x["score"])
|
||||
|
||||
if x["score"] > x["score_rate"]:
|
||||
raise ValueError(
|
||||
f"score is higher than score_rate ({x['score']} > {x['score_rate']}) for {x}"
|
||||
)
|
||||
|
||||
return int(ceil(x["score"] / x["score_rate"] * level_max))
|
||||
|
||||
|
||||
# -----------------------------
|
||||
# Reglages pour 'vim'
|
||||
# vim:set autoindent expandtab tabstop=4 shiftwidth=4:
|
||||
# cursor: 16 del
|
|
@ -0,0 +1,40 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
from math import ceil, floor
|
||||
|
||||
|
||||
def round_with_base(x, base=0.5):
|
||||
"""Round to a multiple of base
|
||||
|
||||
:example:
|
||||
>>> round_with_base(1.33, 0.1)
|
||||
1.3
|
||||
>>> round_with_base(1.33, 0.2)
|
||||
1.4
|
||||
>>> round_with_base(1.33, 1)
|
||||
1
|
||||
>>> round_with_base(1.33, 2)
|
||||
2
|
||||
"""
|
||||
try:
|
||||
prec = len(str(base).split(".")[1])
|
||||
except IndexError:
|
||||
prec = 0
|
||||
return round(base * round(float(x) / base), prec)
|
||||
|
||||
|
||||
def round_half_point(x):
|
||||
"""Round to nearest half point
|
||||
|
||||
:example:
|
||||
>>> round_half_point(1.33)
|
||||
1.5
|
||||
>>> round_half_point(1.1)
|
||||
1.0
|
||||
>>> round_half_point(1.66)
|
||||
1.5
|
||||
>>> round_half_point(1.76)
|
||||
2.0
|
||||
"""
|
||||
return round_with_base(x, base=0.5)
|
|
@ -1,205 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# encoding: utf-8
|
||||
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
from math import ceil, floor
|
||||
from .config import COLUMNS, VALIDSCORE
|
||||
|
||||
# Values manipulations
|
||||
|
||||
|
||||
def round_half_point(val):
|
||||
try:
|
||||
return 0.5 * ceil(2.0 * val)
|
||||
except ValueError:
|
||||
return val
|
||||
except TypeError:
|
||||
return val
|
||||
|
||||
|
||||
def score_to_mark(x):
|
||||
""" Compute the mark
|
||||
|
||||
if the item is leveled then the score is multiply by the score_rate
|
||||
otherwise it copies the score
|
||||
|
||||
:param x: dictionnary with COLUMNS["is_leveled"], COLUMNS["score"] and COLUMNS["score_rate"] keys
|
||||
|
||||
>>> d = {"Eleve":["E1"]*6 + ["E2"]*6,
|
||||
... COLUMNS["score_rate"]:[1]*2+[2]*2+[2]*2 + [1]*2+[2]*2+[2]*2,
|
||||
... COLUMNS["is_leveled"]:[0]*4+[1]*2 + [0]*4+[1]*2,
|
||||
... COLUMNS["score"]:[1, 0.33, 2, 1.5, 1, 3, 0.666, 1, 1.5, 1, 2, 3],
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> score_to_mark(df.loc[0])
|
||||
1.0
|
||||
>>> score_to_mark(df.loc[10])
|
||||
1.3333333333333333
|
||||
"""
|
||||
# -1 is no answer
|
||||
if x[COLUMNS["score"]] == -1:
|
||||
return 0
|
||||
|
||||
if x[COLUMNS["is_leveled"]]:
|
||||
if x[COLUMNS["score"]] not in [0, 1, 2, 3]:
|
||||
raise ValueError(f"The evaluation is out of range: {x[COLUMNS['score']]} at {x}")
|
||||
return round_half_point(x[COLUMNS["score"]] * x[COLUMNS["score_rate"]] / 3)
|
||||
|
||||
if x[COLUMNS["score"]] > x[COLUMNS["score_rate"]]:
|
||||
raise ValueError(
|
||||
f"The score ({x['score']}) is greated than the rating scale ({x[COLUMNS['score_rate']]}) at {x}"
|
||||
)
|
||||
return x[COLUMNS["score"]]
|
||||
|
||||
|
||||
def score_to_level(x):
|
||||
""" Compute the level (".",0,1,2,3).
|
||||
|
||||
:param x: dictionnary with COLUMNS["is_leveled"], COLUMNS["score"] and COLUMNS["score_rate"] keys
|
||||
|
||||
>>> d = {"Eleve":["E1"]*6 + ["E2"]*6,
|
||||
... COLUMNS["score_rate"]:[1]*2+[2]*2+[2]*2 + [1]*2+[2]*2+[2]*2,
|
||||
... COLUMNS["is_leveled"]:[0]*4+[1]*2 + [0]*4+[1]*2,
|
||||
... COLUMNS["score"]:[1, 0.33, np.nan, 1.5, 1, 3, 0.666, 1, 1.5, 1, 2, 3],
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> score_to_level(df.loc[0])
|
||||
3
|
||||
>>> score_to_level(df.loc[1])
|
||||
1
|
||||
>>> score_to_level(df.loc[2])
|
||||
'na'
|
||||
>>> score_to_level(df.loc[3])
|
||||
3
|
||||
>>> score_to_level(df.loc[5])
|
||||
3
|
||||
>>> score_to_level(df.loc[10])
|
||||
2
|
||||
"""
|
||||
# negatives are no answer or negatives points
|
||||
if x[COLUMNS["score"]] <= -1:
|
||||
return np.nan
|
||||
|
||||
if x[COLUMNS["is_leveled"]]:
|
||||
return int(x[COLUMNS["score"]])
|
||||
|
||||
return int(ceil(x[COLUMNS["score"]] / x[COLUMNS["score_rate"]] * 3))
|
||||
|
||||
|
||||
# DataFrame columns manipulations
|
||||
|
||||
|
||||
def compute_mark(df):
|
||||
""" Add Mark column to df
|
||||
|
||||
:param df: DataFrame with COLUMNS["score"], COLUMNS["is_leveled"] and COLUMNS["score_rate"] columns.
|
||||
|
||||
>>> d = {"Eleve":["E1"]*6 + ["E2"]*6,
|
||||
... COLUMNS["score_rate"]:[1]*2+[2]*2+[2]*2 + [1]*2+[2]*2+[2]*2,
|
||||
... COLUMNS["is_leveled"]:[0]*4+[1]*2 + [0]*4+[1]*2,
|
||||
... COLUMNS["score"]:[1, 0.33, 2, 1.5, 1, 3, 0.666, 1, 1.5, 1, 2, 3],
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> compute_mark(df)
|
||||
0 1.00
|
||||
1 0.33
|
||||
2 2.00
|
||||
3 1.50
|
||||
4 0.67
|
||||
5 2.00
|
||||
6 0.67
|
||||
7 1.00
|
||||
8 1.50
|
||||
9 1.00
|
||||
10 1.33
|
||||
11 2.00
|
||||
dtype: float64
|
||||
"""
|
||||
return df[[COLUMNS["score"], COLUMNS["is_leveled"], COLUMNS["score_rate"]]].apply(
|
||||
score_to_mark, axis=1
|
||||
)
|
||||
|
||||
|
||||
def compute_level(df):
|
||||
""" Add Mark column to df
|
||||
|
||||
:param df: DataFrame with COLUMNS["score"], COLUMNS["is_leveled"] and COLUMNS["score_rate"] columns.
|
||||
|
||||
>>> d = {"Eleve":["E1"]*6 + ["E2"]*6,
|
||||
... COLUMNS["score_rate"]:[1]*2+[2]*2+[2]*2 + [1]*2+[2]*2+[2]*2,
|
||||
... COLUMNS["is_leveled"]:[0]*4+[1]*2 + [0]*4+[1]*2,
|
||||
... COLUMNS["score"]:[np.nan, 0.33, 2, 1.5, 1, 3, 0.666, 1, 1.5, 1, 2, 3],
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> compute_level(df)
|
||||
0 na
|
||||
1 1
|
||||
2 3
|
||||
3 3
|
||||
4 1
|
||||
5 3
|
||||
6 2
|
||||
7 3
|
||||
8 3
|
||||
9 2
|
||||
10 2
|
||||
11 3
|
||||
dtype: object
|
||||
"""
|
||||
return df[[COLUMNS["score"], COLUMNS["is_leveled"], COLUMNS["score_rate"]]].apply(
|
||||
score_to_level, axis=1
|
||||
)
|
||||
|
||||
|
||||
def compute_normalized(df):
|
||||
""" Compute the normalized mark (Mark / score_rate)
|
||||
|
||||
:param df: DataFrame with "Mark" and COLUMNS["score_rate"] columns
|
||||
|
||||
>>> d = {"Eleve":["E1"]*6 + ["E2"]*6,
|
||||
... COLUMNS["score_rate"]:[1]*2+[2]*2+[2]*2 + [1]*2+[2]*2+[2]*2,
|
||||
... COLUMNS["is_leveled"]:[0]*4+[1]*2 + [0]*4+[1]*2,
|
||||
... COLUMNS["score"]:[1, 0.33, 2, 1.5, 1, 3, 0.666, 1, 1.5, 1, 2, 3],
|
||||
... }
|
||||
>>> df = pd.DataFrame(d)
|
||||
>>> df["Mark"] = compute_marks(df)
|
||||
>>> compute_normalized(df)
|
||||
0 1.00
|
||||
1 0.33
|
||||
2 1.00
|
||||
3 0.75
|
||||
4 0.33
|
||||
5 1.00
|
||||
6 0.67
|
||||
7 1.00
|
||||
8 0.75
|
||||
9 0.50
|
||||
10 0.67
|
||||
11 1.00
|
||||
dtype: float64
|
||||
"""
|
||||
return df[COLUMNS["mark"]] / df[COLUMNS["score_rate"]]
|
||||
|
||||
|
||||
# Postprocessing question scores
|
||||
|
||||
|
||||
def pp_q_scores(df):
|
||||
""" Postprocessing questions scores dataframe
|
||||
|
||||
:param df: questions-scores dataframe
|
||||
:return: same data frame with mark, level and normalize columns
|
||||
"""
|
||||
assign = {
|
||||
COLUMNS["mark"]: compute_mark,
|
||||
COLUMNS["level"]: compute_level,
|
||||
COLUMNS["normalized"]: compute_normalized,
|
||||
}
|
||||
return df.assign(**assign)
|
||||
|
||||
|
||||
# -----------------------------
|
||||
# Reglages pour 'vim'
|
||||
# vim:set autoindent expandtab tabstop=4 shiftwidth=4:
|
||||
# cursor: 16 del
|
|
@ -2,17 +2,7 @@
|
|||
# encoding: utf-8
|
||||
|
||||
import click
|
||||
from pathlib import Path
|
||||
import yaml
|
||||
import sys
|
||||
import papermill as pm
|
||||
from datetime import datetime
|
||||
|
||||
CONFIGPATH = "recoconfig.yml"
|
||||
|
||||
with open(CONFIGPATH, "r") as config:
|
||||
config = yaml.load(config, Loader=yaml.FullLoader)
|
||||
|
||||
from recopytex.dashboard.index import app as dash
|
||||
|
||||
@click.group()
|
||||
def cli():
|
||||
|
@ -20,62 +10,9 @@ def cli():
|
|||
|
||||
|
||||
@cli.command()
|
||||
def print_config():
|
||||
click.echo(f"Config file is {CONFIGPATH}")
|
||||
click.echo("It contains")
|
||||
click.echo(config)
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.argument("csv_file")
|
||||
def report(csv_file):
|
||||
csv = Path(csv_file)
|
||||
if not csv.exists():
|
||||
click.echo(f"{csv_file} does not exists")
|
||||
sys.exit(1)
|
||||
if csv.suffix != ".csv":
|
||||
click.echo(f"{csv_file} has to be a csv file")
|
||||
sys.exit(1)
|
||||
|
||||
csv_file = Path(csv_file)
|
||||
tribe_dir = csv_file.parent
|
||||
csv_filename = csv_file.name.split(".")[0]
|
||||
|
||||
assessment = str(csv_filename).split("_")[-1].capitalize()
|
||||
date = str(csv_filename).split("_")[0]
|
||||
try:
|
||||
date = datetime.strptime(date, "%y%m%d")
|
||||
except ValueError:
|
||||
date = None
|
||||
|
||||
tribe = str(tribe_dir).split("/")[-1]
|
||||
|
||||
template = Path(config["templates"]) / "tpl_evaluation.ipynb"
|
||||
|
||||
dest = Path(config["output"]) / tribe / csv_filename
|
||||
dest.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
click.echo(f"Building {assessment} ({date:%d/%m/%y}) report")
|
||||
pm.execute_notebook(
|
||||
str(template),
|
||||
str(dest / f"{assessment}.ipynb"),
|
||||
parameters=dict(
|
||||
tribe=tribe, assessment=assessment, date=f"{date:%d/%m/%y}", csv_file=str(csv_file.absolute())
|
||||
),
|
||||
)
|
||||
|
||||
# with open(csv_file.parent / "description.yml") as f:
|
||||
# tribe_desc = yaml.load(f, Loader=yaml.FullLoader)
|
||||
|
||||
# template = Path(config["templates"]) / "tpl_student.ipynb"
|
||||
# dest = Path(config["output"]) / tribe / csv_filename / "students"
|
||||
# dest.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# for st in tribe_desc["students"]:
|
||||
# click.echo(f"Building {st} report on {assessment}")
|
||||
# pm.execute_notebook(
|
||||
# str(template),
|
||||
# str(dest / f"{st}.ipynb"),
|
||||
# parameters=dict(tribe=tribe, student=st, source=str(tribe_dir.absolute())),
|
||||
# )
|
||||
@click.option("--debug", default=0, help="Debug mode for dash")
|
||||
def dashboard(debug):
|
||||
dash.run_server(debug=bool(debug))
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
|
|
|
@ -1,69 +1,4 @@
|
|||
ansiwrap==0.8.4
|
||||
attrs==19.1.0
|
||||
backcall==0.1.0
|
||||
bleach==3.1.0
|
||||
certifi==2019.6.16
|
||||
chardet==3.0.4
|
||||
Click==7.0
|
||||
colorama==0.4.1
|
||||
cycler==0.10.0
|
||||
decorator==4.4.0
|
||||
defusedxml==0.6.0
|
||||
entrypoints==0.3
|
||||
future==0.17.1
|
||||
idna==2.8
|
||||
importlib-resources==1.0.2
|
||||
ipykernel==5.1.1
|
||||
ipython==7.7.0
|
||||
ipython-genutils==0.2.0
|
||||
ipywidgets==7.5.1
|
||||
jedi==0.14.1
|
||||
Jinja2==2.10.1
|
||||
jsonschema==3.0.2
|
||||
jupyter==1.0.0
|
||||
jupyter-client==5.3.1
|
||||
jupyter-console==6.0.0
|
||||
jupyter-core==4.5.0
|
||||
jupytex==0.0.3
|
||||
kiwisolver==1.1.0
|
||||
MarkupSafe==1.1.1
|
||||
matplotlib==3.1.1
|
||||
mistune==0.8.4
|
||||
nbconvert==5.5.0
|
||||
nbformat==4.4.0
|
||||
notebook==6.0.0
|
||||
numpy==1.17.0
|
||||
pandas==0.25.0
|
||||
pandocfilters==1.4.2
|
||||
papermill==1.0.1
|
||||
parso==0.5.1
|
||||
pexpect==4.7.0
|
||||
pickleshare==0.7.5
|
||||
prometheus-client==0.7.1
|
||||
prompt-toolkit==2.0.9
|
||||
ptyprocess==0.6.0
|
||||
Pygments==2.4.2
|
||||
pyparsing==2.4.2
|
||||
pyrsistent==0.15.4
|
||||
python-dateutil==2.8.0
|
||||
pytz==2019.2
|
||||
PyYAML==5.1.2
|
||||
pyzmq==18.0.2
|
||||
qtconsole==4.5.2
|
||||
-e git+git_opytex:/lafrite/recopytex.git@e9a8310f151ead60434ae944d726a2fd22b23d06#egg=Recopytex
|
||||
requests==2.22.0
|
||||
scipy==1.3.0
|
||||
seaborn==0.9.0
|
||||
Send2Trash==1.5.0
|
||||
six==1.12.0
|
||||
tenacity==5.0.4
|
||||
terminado==0.8.2
|
||||
testpath==0.4.2
|
||||
textwrap3==0.9.2
|
||||
tornado==6.0.3
|
||||
tqdm==4.32.2
|
||||
traitlets==4.3.2
|
||||
urllib3==1.25.3
|
||||
wcwidth==0.1.7
|
||||
webencodings==0.5.1
|
||||
widgetsnbextension==3.5.1
|
||||
pandas
|
||||
click
|
||||
papermill
|
||||
prompt_toolkit
|
||||
|
|
|
@ -0,0 +1,69 @@
|
|||
ansiwrap
|
||||
attrs
|
||||
backcall
|
||||
bleach
|
||||
certifi
|
||||
chardet
|
||||
Click
|
||||
colorama
|
||||
cycler
|
||||
decorator
|
||||
defusedxml
|
||||
entrypoints
|
||||
future
|
||||
idna
|
||||
importlib-resources
|
||||
ipykernel
|
||||
ipython
|
||||
ipython-genutils
|
||||
ipywidgets
|
||||
jedi
|
||||
Jinja2
|
||||
jsonschema
|
||||
jupyter
|
||||
jupyter-client
|
||||
jupyter-console
|
||||
jupyter-core
|
||||
jupytex
|
||||
kiwisolver
|
||||
MarkupSafe
|
||||
matplotlib
|
||||
mistune
|
||||
nbconvert
|
||||
nbformat
|
||||
notebook
|
||||
numpy
|
||||
pandas
|
||||
pandocfilters
|
||||
papermill
|
||||
parso
|
||||
pexpect
|
||||
pickleshare
|
||||
prometheus-client
|
||||
prompt-toolkit
|
||||
ptyprocess
|
||||
Pygments
|
||||
pyparsing
|
||||
pyrsistent
|
||||
python-dateutil
|
||||
pytz
|
||||
PyYAML
|
||||
pyzmq
|
||||
qtconsole
|
||||
-e git+git_opytex:/lafrite/recopytex.git@e9a8310f151ead60434ae944d726a2fd22b23d06#egg=Recopytex
|
||||
requests
|
||||
scipy
|
||||
seaborn
|
||||
Send2Trash
|
||||
six
|
||||
tenacity
|
||||
terminado
|
||||
testpath
|
||||
textwrap3
|
||||
tornado
|
||||
tqdm
|
||||
traitlets
|
||||
urllib3
|
||||
wcwidth
|
||||
webencodings
|
||||
widgetsnbextension
|
|
@ -0,0 +1,13 @@
|
|||
---
|
||||
source: ./example
|
||||
output: ./output
|
||||
templates: templates/
|
||||
|
||||
tribes:
|
||||
Tribe1:
|
||||
name: Tribe1
|
||||
type: Type1
|
||||
students: tribe1.csv
|
||||
Tribe2:
|
||||
name: Tribe2
|
||||
students: tribe2.csv
|
Loading…
Reference in New Issue