marcduerst.com — Notes on Code & Cloud

AI-Assisted Software Product Development — A Practical Field Report

Sun, 05 Apr 2026 00:00:00 GMT

Since the beginning of 2026 I’ve been doing more and more AI-assisted software development. The models reached a level where they are genuinely helpful — not perfect, but useful enough to change how I work every day. To explore not just the coding assistance but the full software product development cycle, I started building feldnah.ch — a SaaS application. From idea to business concept, requirements engineering, planning, refinement, development, deployment, security reviews, and marketing. I wanted to understand how future software product teams might look and act.

The following are my findings so far.

Spec-Driven Development is Back

If you’ve been in the industry long enough, you remember spec-driven development. BDD with Gherkin, SpecFlow, custom step definitions — it was a lot of ceremony. The idea was right but the tooling overhead killed it for many teams.

With AI-assisted development, spec-driven is making a comeback — but in a much more practical form:

You don’t need full specs. Write down the important points as bullet points. Then iterate with the AI on the final specification. The AI fills in the gaps you didn’t think of, and you correct what it got wrong.
Natural language, not DSLs. Specs are written in Markdown — no Gherkin syntax, no custom parsers, no SpecFlow bindings. GitHub, GitLab, and any editor renders them nicely as formatted documents.
No implementation from scratch. You don’t need to type every line of code. The spec becomes the input for AI-assisted implementation.

This is a fundamental shift: specs are no longer just documentation that gets outdated. They become the living input for your AI-powered development workflow.

Build a Machine-Readable Briefing

One of the most impactful things you can do is build a structured context that AI tools can read whenever they work on your project. I think of it as a “briefing” — everything the AI needs to know about your product, your standards, and your way of working.

This includes:

Business domain knowledge — what the product does and why
Corporate design — tone of voice, visual language, color schemes
User interaction guidelines — how your app should behave
Technology stack and priorities — what you use and what you prefer
Coding and pattern guidelines — e.g., “we do vertical slices combined with CQRS”

Keep it concise. Don’t be verbose — the AI reads it every time. Every unnecessary sentence burns tokens and dilutes the signal.

A Folder Structure That Works

Here’s the structure I’m currently using. It’s not set in stone — it evolves with every repo I work on. But it’s a solid starting point:

specs/
├── concept.md                   # Core business concepts, USPs, principles
├── architecture.md              # Technical architecture, tech stack, patterns
├── ubiquitous_language.md       # Shared terminology (DDD-inspired)
├── userinterface_guidelines.md  # UI/UX rules and guidelines
├── backlog.md                   # Human-written ideas and backlog items
├── flows/                       # Important user flows
├── stories/                     # One markdown file per story
│   ├── _template.md             # Template for new stories
│   └── done/                    # Archive of completed stories
└── adrs/                        # Architecture decision records
    └── _template.md             # Template for new ADRs

Your README.md should be a concise summary of the most important things — think of it as the landing page of your repo. It’s the first thing people see when they browse your repository or open a PR review on GitHub. Place links to the files in /specs for deeper reading.

Then configure your AI tools (like CLAUDE.md for Claude Code) to reference README.md and the /specs folder as context.

Make the AI Assistant Yours

You can brief the AI on how to behave and respond to you. A colleague told me he configured his Claude to yell back at him or respond in Swiss German when in “yelling mode.” This gives the AI a more personal touch — it makes those long AI-assisted sessions feel less like commanding a dumb machine all day.

I personally switched back to a default conversational mode. I found myself reading responses seriously until I realized the AI was just joking — it slightly increased my mental load. But try it. Some people love it.

This is where it gets practical. Here’s my actual workflow for going from idea to production. It works very well as a solo developer, though it would need adaptation for larger teams.

Capture the idea — Write rough ideas as headings in backlog.md with bullet points. Quick, unstructured, just getting thoughts down.
Generate the story — Prompt: “Write a story for backlog item X.” The AI uses stories/_template.md, respects my specs, ADRs, and past stories. The result is usually more detailed than what a human team would write — and that’s intentional. A good story is key for a good implementation.
Review and iterate the story — Review, edit manually or prompt the AI to fix specific aspects. When solid, commit backlog.md and all story files into Git — they’re part of the codebase.
Implement — Prompt: “Story X looks good, implement it.” The AI reads the story, specs, existing code, and writes the implementation.
Review and test — Quick code review of what changed, focusing on the important parts and unit test coverage. Then manually test every user-facing change. No exceptions.
Iterate — Using AI prompts or, rarely, by coding manually.
Document decisions — If architectural decisions were made, have the AI write a new ADR. If specs need updating, tell the AI — or it does it automatically because it’s part of my story template.
Close and ship — Prompt: “Close story X.” This writes the changelog, updates the story status to done, removes it from the backlog, and moves the story file to /specs/stories/done. Then “commit and push” triggers CI/CD and the feature is live.

A Note on Code Reviews

How I review AI-generated code evolved over time. In the beginning I reviewed every single line. Now I focus on the big picture and the critical parts. I don’t spend much time reviewing CSS changes or straightforward UI code.

I also adjusted my mindset on code quality: not every line needs to be diamond-level from day one. Core logic and persistence absolutely should be. But the UI layer can have some redundancy — I can refactor it all at once later using AI. This would have been heresy for me two years ago.

Beyond Coding: AI in the Product Lifecycle

AI-Assisted Market Analytics

I use AI to query various aspects of the business around my product. The specs provide context — they’re like a persisted memory for the AI. I always ask for sources to validate business numbers. If your market analysis is based on speculation, your product will hit the wall.

AI-Assisted Security Reviews

AI-assisted security reviews are remarkably effective when you have good specs. The AI understands what’s important for this specific product and at what security level you need to operate.

I had a genuine wow moment when I asked the AI to calculate worst-case scenarios for my SaaS app. “What happens if my app gets DDoS’ed? What will be the costs?” Claude Code produced a comprehensive analysis with serious calculations for various scenarios. I was already in good shape (it’s not my first cloud app), but it found interesting gaps that I then closed. The cost calculations were surprisingly accurate — I double-checked them against actual cloud pricing.

Always verify security findings. But as a starting point, it’s incredibly efficient.

AI-Assisted Operations Debugging

In my day job at Switzerland’s biggest e-commerce site, I started using AI for production issue debugging. My reasoning: if there’s one thing these models are good at, it’s finding patterns and correlations in large amounts of data.

I connected various data sources:

Git via CLI — for looking up implementations including infrastructure-as-code
GitHub via gh CLI — for analyzing deployments to production
Datadog via MCP — for standard and custom metrics
Google Cloud via gcloud CLI — for querying real infrastructure and GCP logs

I also wrote a docs/monitoring.md with all monitoring links: dashboards, metrics, notebooks, alerts. Then I created a skill (a reusable prompt) that tells the AI how to drill down production issues, references the monitoring doc and specs, and outputs a readable Markdown report.

I iterate on that report like a living document — it becomes the persisted context for the investigation.

What genuinely surprised me was the patterns the AI found — correlations I wouldn’t have spotted as quickly myself. It also figured out creative ways to query the information it needed, adapting well to our specific tooling and data structures. When standard queries weren’t enough, it wrote Python scripts on the fly for more complex data gathering and analysis.

That said, it’s critical to double-check every finding. You need the expertise yourself — you cannot blindly rely on AI for production debugging. I always include something like “no speculations, only hard and validated facts, double-check numbers if possible” in my prompts. Without that, the AI tends to fill gaps with plausible-sounding guesses — which is the last thing you want in a production incident.

AI-Assisted Cloud Cost Debugging

When cloud costs spike unexpectedly, you want to know why fast. This can be straightforward or extremely complex and time-consuming.

I use the same approach as operations debugging, but with an additional data source: cost reports (via gcloud CLI or CSV exports from the GCP console).

I was genuinely surprised at how well Claude Code correlated cost increases with operational events and traced complex, accumulated effects along a timeline. It can reduce a massive amount of time and effort that would otherwise go into manually cross-referencing billing data, metrics, and deployment events. But you need cloud cost expertise yourself to quickly judge when the AI is going off track — and it will, especially with complex multi-service billing.

More Ideas for AI Assistance

The list of things AI can assist with keeps growing. Here are some I’ve explored or seen in practice:

Implement UIs from Figma designs — AI reads existing Figma documents via MCP and generates the frontend code to match
Generate social media content like Instagram reels using Canva MCP
Web UI prototypes using Gemini’s web preview capabilities
UX documentation — tone of voice, color schemas, personas
AppStore screenshot updates — stitching new screenshots into marketing images
Release notes and changelogs — generated from commit history and stories, adapted to the target audience (users, not developers)
Dependency updates — update libraries and frameworks including migrating breaking changes across repositories
Concept drafts from Confluence and Jira using connectors/MCP
UX analysis — AI reviews your interface and suggests improvements
SEO analysis and optimization right in the code
Social media post optimization — tailored per platform (Instagram is different from LinkedIn)
Automated user testing — define different personas and their behavior, then let AI test your software automatically with each of them
Regional research — e.g., “find all farmers with farm shops within 20km that have online ordering, and compare their features”
Translation automation in web and native apps
Full native app releases — build, translate, publish to Apple, generate “what’s new” tailored to your audience in all languages, and more. Goes way beyond static CI/CD pipelines because it can adapt

Thoughts on What This Means

I want to close with some broader reflections. These are things I think about a lot, and I don’t have all the answers.

The Industry Can’t Ignore This

AI tooling has reached a level where the software industry cannot afford to look away. Companies that don’t adapt will face serious competitive disadvantages. The same goes for professionals — dismissive hate-speech about AI tools won’t age well. The pragmatic approach is to understand what these tools can and can’t do, and use them where they genuinely help.

Quality Still Requires Expertise

To get professional-quality results, you need to have the expertise. AI amplifies what you bring to the table — if you bring deep knowledge, you get excellent results. If you bring nothing, you get plausible-looking garbage. The human in the loop is not optional. It’s the quality gate.

The Junior Developer Question

This one keeps me up at night. AI handles most of the basic tasks that juniors traditionally learn on. But to work efficiently with AI, you need decent domain and technical knowledge. How do you build that knowledge if AI is doing the doing? Our industry needs to think seriously about mentorship, learning paths, and what “junior” even means in an AI-assisted world.

Organizational Change Beyond Code

This isn’t just about AI-assisted coding. Workflows, processes, and even team structures will change. Product owners will refine differently. QA will test differently. The roles of architect, developer, and operator are blurring. We’re looking at a fundamental shift in how software product teams are composed and how they collaborate.

I’m joining a temporary cross-functional team at Digitec Galaxus that’s exploring exactly these new ways of working. I’m looking forward to learning a lot about future team dynamics and workflows — and I’ll keep you posted.

The Pace and the Uncertainty

I can’t imagine what will be possible in just six months. The pace of improvement is breathtaking. But I suspect it will slow down at some point as things saturate. Every major AI company is running deep financial losses. If this technology persists — and I believe it will — it will likely become more expensive.

The Environmental Cost

The environmental impact of this way of working is significant, and I’m not a fan of this side of the medal. Training and running these models consumes enormous amounts of energy. As an industry, we need to ask: how do we make this sustainable? I don’t have a good answer yet, but ignoring the question isn’t one either.

This post reflects my experience as of early 2026. The tooling and best practices are evolving rapidly. What works today might look different in a few months — but the underlying principles of good specs, human expertise, and critical thinking will remain.

Integrating Machine Learning — A Software Engineer's Guide

Fri, 03 Apr 2026 00:00:00 GMT

This is not a post about how to train models or write Python notebooks. It’s about what I’ve learned as a software engineer working alongside data scientists — how to collaborate effectively, what to watch out for when integrating ML into production, and why the offline/online distinction matters more than most people think.

But lets start with some basics…

What is a Data Scientist

There are two roles sound similar for software engineer: Data Analyst and Data Scientist. The two roles are often confused, but from an engineering perspective they have very different needs and outputs. Think of it in software terms:

Data Analysts

Data analysts are like your monitoring and debugging experts — but for the business. They look at what happened and try to explain why.

They query collected data, build dashboards, spot anomalies, and present findings to stakeholders. Their output is insights and recommendations, not code that runs in production.

In engineering terms: if a data analyst were a developer, they’d be the one reading logs and metrics after an incident, writing the post-mortem, and suggesting what to fix — but not writing the fix themselves.

Example: “Conversion dropped 12% last week. I dug into the funnel data and found that 80% of drop-offs happen on the new checkout page on mobile devices. I recommend we prioritize a mobile UX review.”

Data Scientists

Data scientists are closer to R&D engineers. They don’t just analyze what happened — they build things that predict what will happen or automate decisions. Their output is algorithms, trained ML models, or heuristics that engineering integrates into production.

In engineering terms: if a data scientist were a developer, they’d be the one writing a proof-of-concept for a new feature — except their “feature” is a mathematical model that needs your infrastructure to run.

Example: “We trained a recommendation model that predicts which products a user is likely to buy based on their browsing history. It improves click-through rate by 15% in our offline tests. We need engineering to integrate it into the product page — the model takes a user ID and returns a ranked list of product IDs in ~10ms.”

How Data Science and Engineering Work Together

Data science and software engineering don’t work in isolation. There are three distinct phases where the two disciplines intersect — and each has different needs.

Phase 1: Data Science Research

This is the exploratory phase. Think of it as the data scientist’s playground.

Needs ad-hoc query capabilities on big data
Needs historical data — current data alone is mostly unusable for research; timelines matter
Very individual — every data scientist works differently, with their own tools and workflows

Phase 2: Model Training

A more structured phase where models get trained and validated.

More aligned tooling, possibly standardized across a team or company
Needs query capabilities on big data
Needs historical data (e.g. snapshots from various points in the past)
A more stable, repeatable approach to querying data
Data lineage becomes important: who uses what data, for what purpose, and what are the dependencies

Phase 3: Software Engineering Integration

This is where engineering takes over — integrating the algorithm or ML model into the production system.

Can be offline (batch) or online (real-time) — more on this below
Requires tight alignment on data formats, types, and semantics
Fast database queries: all data including feature data must be loaded in milliseconds
No historical data needed — only current data matters in production
Data access may be charged per query, so optimize for efficient database usage
Smart caching for feature data may be needed to reduce latency and cost
May need to log predictions for future model training (feedback loop)
Operations monitoring, SLA/SLO — are ML predictions working as expected? Latency, error rates, prediction quality

Keys to Good Collaboration

Start early. Data scientists and software engineers need to collaborate already in late research phase. Why? Because integrating some ML features as well as the type of model in production can be a deal-breaker. Often features that sound easy from a data science perspective are hard to implement in engineering — or vice versa. The engineering must also be able to load and use the model on their tech stack.

Get the data right. When engineering integrates an ML model, it’s crucial that the data types and semantics match exactly what the data scientist used during training. If you use two different tech stacks, things get tricky — there can be semantic differences between online and offline data. Or there can simply be bugs. Even if the tech-stack is the same the data-sources may not. Ensure that the data-science database (eg DWH) aligns with the engineering database (online).

Agree on the model format early. Both sides need to be able to work with the chosen type of ML model. Data science needs to train it effectively, while engineering needs to load it and run predictions/inferences in production. From an engineering perspective, what matters is: how big is the model (memory, load time), and do we have libraries to load this model type in our stack? This becomes critical when the tech stacks differ — e.g. data science uses Python/Airflow while engineering runs .NET or Java backends. A model format that’s trivial to export in Python might have no mature loader in your production runtime. Solve this before anyone starts training.

Offline vs. Online: The Most Important Distinction

This is where most of the engineering complexity lives.

Offline (Batch Predictions)

From an engineering perspective, offline means running ML predictions in batch jobs — typically on a schedule.

Latency doesn’t matter — jobs can take minutes or hours
Data freshness doesn’t matter — you work with whatever snapshot is available
Results are pre-calculated and stored for fast lookup by the online system
Stability is secondary — a failed batch can be retried without user impact
Resource costs (memory, storage, compute) still need attention

A practical example: Calculate trending products every 2 hours. Once done, write the results to a fast key-value store where the online system picks them up.

Online (Near Real-Time)

Online means the user is waiting — your service needs to load the model, gather feature-data, run inference, and return a result within a single request.

Engineering challenges:

Total response time often needs to be below 50ms — including feature loading and inference
The ML model must be loaded in memory and ready to serve (cold starts are a killer)
The ML model updates most be loaded while staying full operational
May support A/B testing of ML models — not one but two models must hot loaded in memory
To detect model skew you need some kind of model validation (eg. run a well defined test-case the model is loaded into memory but before you use the model for serving user requests)
Feature data must be queryable in milliseconds — this is often the actual bottleneck, not the inference itself
Data must be kept current via streaming pipelines (e.g. Kafka, Pub/Sub) into a fast store
Everything must be failsafe with graceful fallbacks — if the model fails, the user shouldn’t notice
Must scale horizontally under load
Cloud costs can spike quickly — every query, every byte of storage adds up
Massive tracking data for model inferences need to be collected stored while under full load

When you can’t avoid online predictions:

When real-time data is needed as model input (e.g. products viewed in the current session)
When the number of feature combinations is too large to pre-calculate — you can’t pre-compute every possibility

What engineering typically needs to build:

A feature store with millisecond read latency, fed by streaming data. For a large E-Commerce site like Digitec-Galaxus expect hundreds of GB if not TB’s of data. Collections/tables can contain multi-million documents/rows each.
Model serving infrastructure (load & validation, version, swap models without downtime)
A distributed caching layer for frequently requested feature data (not for prediction results)
Monitoring for prediction latency, error rates, and model quality (drift & skew detection)
Logging of predictions and inputs for future model retraining

Decision Framework: Offline or Online?

Default to offline if you can. Challenge requirements so you don’t need real-time online ML predictions. Two factors push you toward online:

Data freshness — if you need very current data, you can’t pre-calculate every few hours. At some point, you have to do it in real-time.
Granularity — if the model has many features or a feature has a wide range of values, pre-calculating all combinations becomes infeasible. But if the model only needs a product ID and an age group, you can pre-calculate rows like (product_id, age_group) → recommendation — even millions of rows are fine for today’s cloud databases.

Tips for Online ML

Don’t use slow or costly models in online scenarios
- Challenge to use an efficient model for the job; not the fanciest newest thing
- Quantise the model for production (data-scientist can do this after they trained the model but often you need to tell them to do so)
Custom ML models can be quantized — this reduces model size dramatically and makes inference fast
Remember: the bottleneck is usually data loading, not inference
Design your feature store for millisecond lookups with streaming updates

Drift vs. Skew — Know the Difference

Two terms you’ll hear from data scientists that matter for engineering:

Model drift means the model’s predictions get worse over time because the real world changed. The patterns it learned during training no longer match reality. Example: a recommendation model trained on pre-pandemic shopping behaviour performs poorly after consumer habits shifted. Engineering’s job: monitor prediction quality and trigger retraining when metrics degrade.

Model skew is a mismatch between training data and serving data — same point in time, but the data looks different due to technical inconsistencies. Example: during training the feature “user_age” was a float, but in production it arrives as an integer. Engineering’s job: ensure the feature pipeline produces identical data formats and semantics in both offline training and online serving.

In short: drift = the world changed, skew = your pipeline has a bug.

What Does “Quantize a Model” mean?

ML models store their learned knowledge as millions of numerical weights — typically as 32-bit floating point numbers. Quantization reduces the precision of these weights, for example from 32-bit floats to 8-bit integers. The model file gets dramatically smaller (often 4x), loads faster, and runs inference quicker — with only a small, often negligible loss in prediction accuracy.

Why this matters for engineering: a quantized model uses less memory, starts faster (critical for scaling and cold starts), and is cheaper to serve. Data scientists can quantize after training, but they often don’t think about it unless you ask. Make it part of your handover checklist.

Closing Thoughts

Neither data science nor engineering can deliver ML value alone. Collaborate early, push back when something won’t work in production, and stay in regular contact — not just at handover points. A tricky feature for engineering might have a simpler alternative that data science can explore while still in research. The best ML systems are built by teams where both sides understand each other’s constraints.

Tech stack for hosting small apps on Azure for free

Sun, 08 Oct 2023 00:00:00 GMT

I’m sometimes doing web apps for hobby/personal projects which I like to host online just very low cost or for free. I have the following set-up in mind for quite some time and finally took the time to explore it and got it run. In this post I write my findings.

Summary

With Azure Functions and Azure Storage Accounts combined, you get a powerful and very cheap or free stack for smaller projects running SPA’s, .Net backends and storing various kinds of data.

As always, it needs tome time to figure the nasty details on how to connect the building blocks. Also have a look at the limitations. You also must be open on new tech and not just require good old MS-SQL server.

My requirements

Basically my wish list from the technical side with focus on web-frontend and backend. In the future, maybe native mobile-apps will join.

Backend
- Modern .Net (Version 6/7/x)
- Supported facades: API’s (HTTP), scheduled Jobs (Timer), message handlers (Queues/Events)
- Support storages: no-SQL, blobs, queues, optional: SQL/relational, optional: others
Frontend
- Support for SPA frameworks (Rect/Next or others)
- Custom domains
- Free to use any CSS-frameworks, component-libraries etc.
- Frontend can call the backend API (HTTPS, CORS, TLS, …)
Mobile Apps
- Possibility to communicate to the backend for native mobile apps in the future

Choose tech-stack

To host projects with the above stack with low or no costs, I came up with the following:

Repository: GitHub
- Free for personal, small or open-source projects
CI / CD: GitHub Actions
- Some usage for free included in GitHub repo.
Backend: Azure Serverless Function app(s) written in .Net C# 7
- Supports HTTP, timers, queues, storage and others as triggers
- In the consumption plan, there is a decent compute amount included for free: 400’000 GB-s.
  Tip: Set a daily usage consumption on max. 13’000 GB-s/day to make sure you stay in free tier. Change this to our app and budget needs.
Data: Azure Table Storage (part of Azure Storage Account), an option is Azure Cosmos DB with its free tier (<1000 RU/s, <25 GB)
Blobs/Fles: Azure Blob Storage (part of Azure Storage Account)
Messaging: Azure Storage Queue (part of Azure Storage Account)
Frontend: Azure Blob Storage Static Website (part of Azure Storage Account)
Telemetry/Monitoring: Azure Application Insights (Azure Monitor & Log-Analytics)

Azure Storage Accounts

All of my storage requirements for small projects are covered by Azure Storage Accounts. This service is very cheap (eg. €0.017 per GB/moth for Hot-Storage, €0.009 per GB/moth for Cold-Storage)! That’s basically free for most small projects.

Azure Storage Accounts is the home for:

Blob storage: storing files/buckets/etc. in hierarchical containers. Optionally, you can also use file-sharing to upload and download files from Windows Explorer / Mac Finder – maybe interesting for some special use-cases.
Table storage: simple and highly scalable NoSQL data, with some restrictions on indexing and querying. Check the limitations and best-practice for partition- and row-keys before using it. Optionally supported by Azure Functions with input- and output-bindings.
Queue: Simple message-queues. Supported by Azure Functions as queue-/message-triggers.

Azure Static Web Apps

Because the combination of Azure Functions and Azure Storage Accounts is very powerful to host backends and SPA’s, Microsoft is building a dedicated service around this called Azure Static Web Apps. It simplifies the configurations and gives some advanced options but also introduce some limitations.

A bonus is that you also get a Node-JS server where you can do things like server-side rendering.

Limitations

One limitation that hit me, was that the Azure Functions only supporting HTTP triggers; timers, queues and other triggers are not (yet?) supported.

Another limitation: the free plan does not support “bring your own Functions backend” and therefore you can not work around the above limit by bringing our own Azure Functions instance.

None of these limitations you have when using your own custom Azure Function instance.

Resulting Azure Resource Group

To give you an idea, here is what my Azure Resource Group looks like. My PoC is analyzing free parking spaces in the City of Zürich. Don’t look too close on my naming – it’s just a proof of concept 😉.

Some points of interest:

Custom Azure Functions App to get timer support
Custom Azure Storage Account to host my tables and the static website (SPA). Separate from the Functions own internal Storage Account.
Standard Application Insights instance

Implementation

I don’t like to go into every detail here, but some tips on things I struggled with.

Secure access to data

To keep the data save, I restricted the access to my Azure Table Storage using an Azure Managed Identity. It needed the “Storage Table Data Contributor” role.

Static web app

In Azure Storage Account there is a menu-item called “Static Website”. Enable it to get the required “$web” blob container. Place your SPA files and assets in this container – it is your web-root. See below for my GitHub Actions workflow on how you deploy it into this container.

CORS (Browser Security)

It took me a while until I figured where to add the CORS headers. They are needed because with this setup, your backend and your web-hosting (SPA) use different domains (=origins).

In the Azure Functions app, there is “CORS” menu-item. Add the domain / URL of your SPA app.

In the blob storage (static website) there are CORS headers too. You’ll find it under “Resource Sharing (CORS)”. Add the URL / domain of your Azure Functions backend in there.

CI / CD with GitHub Actions

Deploying the Azure Functions app is mostly using the generated GitHub Actions YAML file with a few tweaks like the path filters in the trigger. I set them so only backend-changes or changes to the backends CI/CD, trigger a backend build&deploy.

name: Backend - Build and deploy

on:
  push:
    branches:
      - main
    paths:
      - src/ParkingZuerichAnalytics/**
      - .github/workflows/backend_parkingzuerichanalytics.yml
  workflow_dispatch:

env:
  AZURE_FUNCTIONAPP_PACKAGE_PATH: './src/ParkingZuerichAnalytics/ParkingZuerichAnalytics.DataGathering'
  DOTNET_VERSION: '7.0.x'

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: 'Checkout GitHub Action'
        uses: actions/checkout@v2

      - name: Setup DotNet ${{ env.DOTNET_VERSION }} Environment
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: 'Resolve Project Dependencies Using Dotnet'
        shell: bash
        run: |
          pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
          dotnet build --configuration Release --output ./output
          popd

      - name: 'Run Azure Functions Action'
        uses: Azure/functions-action@v1
        id: fa
        with:
          app-name: 'ParkingZuerichAnalytics'
          slot-name: 'Production'
          package: '${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}/output'
          publish-profile: ${{ secrets.AZUREAPPSERVICE_PUB.... }}

Deploying the NextJS SPA was a little more effort. Important is the fact that you don’t get a Node server for your frontend. So you have to use the static export of NextJS. Depending on the lifetime of your data you also need to ‘use client’ so the client (Browser) will fetch the data, not the NextJS build-process.

For a tutorial on how to deploy the static website, see here.

name: App - Build and deploy

on:
  push:
    branches:
      - main
    paths:
      - src/parkingzuerich-app/**
      - .github/workflows/app_parkingzuerichanalytics.yml
  workflow_dispatch:
    
defaults:
  run:
    working-directory: ./src/parkingzuerich-app

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout GitHub Action
        uses: actions/checkout@v2
        
      - name: Install NodeJS
        uses: actions/setup-node@v1
        with:
          node-version: 18
          
      - name: Install packages
        run: npm install
        
      - name: Build static site
        run: npm run build
        
      - name: Login to Azure
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Upload to blob storage
        uses: azure/CLI@v1
        with:
          inlineScript: |
            az storage blob upload-batch --account-name parkingzuerichapp  --auth-mode key -d '$web' -s ./src/parkingzuerich-app/out

      - name: Logout from Azure
        run: 
          az logout
        if: always()

Alternatives

Hosting Next JS SPA: Vercel has a free plan. Interesting if you use Node for your backend.
Google Cloud (GCP): very cheap options for messaging (Google PubSub) and no-sql data (Google Firestore).

Closing

If you open your mind to new tech, you can get really cheap hosting for your smaller projects. Just stay under the radar to not trigger the expensive limits. Make a PoC if things work for you.

Hope this helps and inspires!

Monitor a pod's container resources in GCP

Wed, 01 Feb 2023 00:00:00 GMT

Recently, we switched to GKE (Google Kubernetes Cluster) “autopilot” with the whole cluster. Autopilot is a feature where Google will manage the required VM’s / Scale-Sets for you. You will not need to specify, monitor, scale or even pay for VM’s and therefore IaaS. Instead, you specify what workloads (services and cronjobs) you have, and how much CPU and memory they need. These are called resource “request”. GKE needs this information to do proper horizontal auto-scaling. You finally pay with autopilot is the total-requested CPU and memory of all your Pods (Instances) – no matter how many VMs and what size of VM’s is needed to run all this.

The problem

How to find the right values on how much CPU and memory each of your workload uses?

If you go to the GCP console – the web-UI of the Google Cloud – you can drill down to your workload details and get something like this:

You see how many vCPU’s all your Pods need together. The red line is what one specified in the deployment YAML files. First though: nice and quick overview.

But this can be miss-leading as it shows the aggregated statistics of all containers instances running in all your pod’s. Especially with autopilot, resource requirements are managed strictly per container.

So the question is, how much CPU and memory does each of your container really use/need? What does one have to specify in the YAML for which container?

The solution

You can monitor the resource-usage of your containers in the Google Metrics Explorer.

CPU is found in Kubernetes Container > Container > CPU usage time. Memory just next to it in Kubernetes Container > Container > Memory usage.

Then a lot of lines will be shown in a diagram: one for each container (not pod) in your cluster:

Not quite that helpful 😁.

Pro-tip: add a filter on “app” which is the name of your workload:

When you get all running containers for the given workload like this and on the right side you see the number of cpu’s for the given container and point in time.

I’d say this is better, but still not perfect. Our pods normally have two or three containers running per pod. For example, our own backend-service and an Istio (mesh) sidecar and sometimes even a Google Cloud SQL proxy. All of these containers are shown in the above diagram as well.

As you have to specify the resource requests per container, you normally like to take a look at a single container type. For example, workload (=app) “CustomerBackend”, container (=container_name) “istio-proxy”.

Pro-tip: add a filter on container-name.

Now the chart looks better, and you see relatively good, what the istio-proxy container of your customer-backend workload really uses:

Compare the value on the Y axis with the request values you have in our yaml or Helm chart and adjust them if needed.

Pro-tip: you can also use other features of the metrics explorer like group, e.g. by container_name. To count the instances running over the time, you can filter by app and group by container_name and then aggregate the count. You can even go further and build your custom GCP dashboard based on the metrics settings with filter variables, so you can use the dashboard to check each of your workloads by using the filter toolbar.

A note on hitting the requested CPU and memory limits:

If the CPU values reach the value of your requested CPU, the container gets suspended for a while by the underlying Linux kernel. Normally this is not helpful and gives you other issues. Especially containers like the istio-proxy (e.g. used by Google Anthos) don’t like to be suspended as they control all the network traffic in and out of your pods. They also control network access for Google services like the GKE meta-data store. This is needed to inject your workload settings (environment variables) into your application containers.

Hitting the memory limit is about what you may expect: classic out of memory errors will occur.

I guess you can imagine all sorts of strange errors coming out of these situations. So don’t just set the values for your application container right, but also for the supporting containers your pod runs.

Summary

It’s hard to get the request values for CPU and memory right, as the container have peaks and lows. But it’s important to keep them running smoothly and on the other hand don’t pay too much for your cloud compute power.

The Google Metrics Explorer is your fried here and offers many metrics and features to nail down your cluster.

If you have Google Anthos (Istio) meshing configured in your cluster, you get even more metrics out of it. You can even go one step further and use KEDA to do event-based auto-scaling of your pods using many more metrics. This all works great with GKE autopilot on our clusters.

Scale and fix a Azure Service Fabric cluster

Thu, 01 Apr 2021 00:00:00 GMT

After having not quite an easy time fixing our production server-cluster manually I like to write a few things down so maybe other or my future self will find these information useful.

Situation

Our production server is a 5 node “Silver” cluster. It currently serves 17 services, 85 replicas. It serves 5 mio HTTP-Requests and about 25k users per day. This means that this system is fairly busy all day as we have users online also during night-times.

This system reported failed Service Fabric Cluster Upgrades because one node did not update (timeout) for some reason. Service Fabric Cluster Upgrades are initiated by Microsoft to update the cluster-runtimes on the machines.

Because we can not RDP into these nodes for security reason (working on this right now) we decided – together with Microsoft Support – to simply replace the “broken” node with a fresh one. That’s where our Service-Fabric manual scaling adventure started.

Learning: Get a healthy cluster after each operation

After you have done an operation on your cluster get a healthy cluster before starting the next operation. Otherwise your cluster can get into serious trouble if problems are start summarise up.

Learning: How to scale-out/-in manually

Every Service-Fabric has its associated Azure VM-Scalue-Set underneath. The scale-set consists of a set of identical VM’s. These VM’s have the Service-Fabric extension installed and that is how Service Fabric Cluster Manager connects and controls the VM’s. Each VM becomes a “node” in the cluster. Its also recommended to have multiple scale-sets per fabric. At least one for the so called “seed” nodes and another for the application-services. We don’t do that (yet) because of simplicity and budget.

If you like to have additional nodes on your Service-Fabric you simply can change the number the number of VM’s in your scale-set and wait some time. It may take of up to an hour before the VM’s are provisioned and become full nodes with our software running on it. If you don’t need the additional power (nodes) anymore you simply can scale the VM scale-set accordingly. You find these scaling option in the Azure Portal on our VM scale-set under “Scaling”.

Here is a more detailed documentation from Microsoft.

Learning: Never ever go below 3 nodes

No matter what the reliability tier is (Bronze, Silver, Gold, Platinum), if you go below 3 nodes your Service-Fabric can completely break and is not accessible at all. We faced this issue several times on our dev/test environments. Lucky us we have scripted our infrastructure so we deleted the scale-set and service-fabric and re-depolyed them from scratch. Otherwise not even the Service Fabric Explorer (SFX) of those fabric’s was accessible anymore.

Learning: Seed nodes promotion

Seed nodes are nodes which some Service Fabric System stuff on it. Depending on our reliability tier Service-Fabric needs 1 (None), 3 (Bronze), 5 (Silver), 7 (Gold) or 9 (Platinum) seed nodes. If a seed-node is un-healthy over about 2 hours or so the Service-Fabric will promote another node to a seed node automatically. So, if you run into this situation you just have to wait a decent time.

Let’s say you did it the hard way and deleted a VM which was a seed-node. Service Fabric will list this node as seed-node with status “down”. You can not delete this node yet as it is a seed node. If you did not already have spin up a fresh VM with scale-out on the scale-set. Then wait up to two hours or so. SF will promote the healthy non-seed node to a seed-node. If this happens you can “Disable (remote data)” (see below how to get this option) and it will be removed from the node list.

Learning: Node management in SFX with “Advanced Mode”

The Azure Portal UI of Service-Fabric is fairly basic. If you like to have more insights or get a UI for advanced operations you need to use the Service Fabric Explorer (SFX). You’ll find a link to it in the Azure Portal of your Service-Fabric.

On the nodes you get a “…” menu which – by default – has only a few options in it. And here is the trick we where told by Microsoft Support: Click on the gear-icon (“Settings”) on the top-right of the screen and activate the “Advanced mode” checkbox.

Now your “…” menu shows a bunch more options especially for the nodes. For example you then can “Disable (remove data)” and other disable options. This instructs SF to disable a node with the given ‘intent’. For example the “remove data” will shut down your services and remove the replicas. Wait a second to check that the node will not run your app/services anymore.

These options are helpful if you like to remove nodes for some reason. If you do this using Power-Shell or Azure CLI you can use the –force option so you don’t have to wait on seed-node timeouts. But again: never ever go below 3 healthy seed-nodes. If you need to remove one, first make sure you spin up new nodes before so after the removal you still have at least 3 healthy seed-nodes.

Learning: SF will not re-balance fault-domain if you scale manually

When you do things like to above operations it can happen, that you cluster-map ist “not balanced” anymore. This means that some of the nodes share upgrade- or fault-domains. You see this in the tab “Cluster Map” of the SFX.

A balanced cluster-map looks like this:

Service-Fabric will not fix this if you scaled manually or if you manually removed nodes. You have to take care of it.

A fault-domain can not be changed easily because it defines where you VM physically lives. To change this you need to get a new VM in place eg. by scale-out again. If you have VM’s in the right spot of your cluster-map you can remove the obsolete VM’s. To do so, first disable them in SFX with the remove-data option. When the replicas are down, you can delete the VM’s in the VM scale-set using Azure Portal or CLI.

Learning: Use supported VM-sizes

When you do a scale-up or scale-down by selecting a new VM-size make sure you choose one that is supported by Service-Fabric otherwise your Service-Fabric can explore so you can not access it anymore and have to start over from scratch (re-create SF and Scale-Set).

The bad thing: there is no safety belt when re-sizing VM’s. You will not get any warning or so but your SF will break afterwards.

Check the Microsoft Documentation here…

Conclusion

There are several things one needs to know (or learn) for operating a production Service Fabric Cluster. If you don’t you can burn down your cluster if you do the wrong thing.

Therefore make sure you have a multi-node test-cluster with 3 nodes or so. Its still not the same as a production cluster with 5 or more nodes (see the fault-domain limitations) but at least you can try most of the things on the test-cluster first.

Order Azure Log Analytics result columns

Tue, 01 Sep 2020 00:00:00 GMT

Problem

When doing a Kusto query in Azure Log Analytics the result set (Grid) gets rebuild every single time a query is executed. The default order of the columns is somewhat random. This normally means that I have to reorder the important columns like level, message etc. manually by dragging them to the first position. If they are not even shown I first need to show them using the column picker drop-down.

This is no fun and I finally found the trick to build my “perfect” query so in don’t have to fiddle with the grid every time.

Solution

One of my favorite query is to query all our logs (union *) for a given Correlation-ID (where) over the past days (ago(…)) and show the most important props as the first columns and then include all other columns / log values (project-reorder) so I can expand a log row and still get all the log properties for this log entry. I order the list using the Timestamp value as this one is more precise then TimeGenerated.

Here is my query:

union *
| where LogProperties_CorrelationId_g == "4869ea08-a9cf-49ed-a281-b5217655b65f"  and Timestamp_t > ago(7d)
| order by Timestamp_t desc nulls last 
| project-reorder TimeGenerated, Type, LogLevel_s, LogMessage_s, LogException_Message_s

Hope this helps others.

Let me know your cool and handy Kusto tricks!

Track SQL data with Azure alerts and dashboard charts

Sun, 21 Jun 2020 00:00:00 GMT

Note: This post originally contained 3 images that could not be recovered during the WordPress migration.

I’m trying already for quite some time to track the result of some SQL query in my Azure Dashboard and even better: trigger Azure Alerts when some condition is reached. See my previous blog posts on Query SQL database using a Azure LogicApp and Show results of SQL query on Azure Dashboard. Thanks to to some ideas from my college we found a better approach to this requirements.

Goal

We have a SQL query which returns us a count. We like to monitor this count in a Azure Dashboard chart and trigger Azure Alerts when they reach a certain value.

Solution

Here is the short version how we finally got in working:

Create a Azure Logic App with a “Schedule” trigger.
Add action after the trigger to query the database (see here).
Tip: To know the structure of the SQL query result for the the next steps, run the Logic App and check the SQL query’s output.
Use “Settings” on the query action and add a “Tracked Property” which logs the value from the query result to the diagnostic log.
Expression in Tracked Properties value I used:
"@int(body('Query_Event_Count')?['resultsets']['Table1'][0]['count'])"
Add a diagnostic setting to log “WorkflowProcess” and “Send to Log Analytics”.
Run the Logic App and check everything is working.
Wait about 5 minutes and check Log Analytics “Azure Diagnostics” for the query action and see if the “Tracked Property” got recorded correctly.
Create a Log Analytics query which query the these tracked values.
AzureDiagnostics | where TimeGenerated > ago(24h) and trackedProperties_countEvents_s != "" | order by TimeGenerated desc nulls first | project CountEvents=trackedProperties_countEvents_s
Use this query to define an Azure Alert Rule with source set to Log Analytics.
Use the same query in a Azure Monitor “Workbook” to query and render a chart or whatever you like to visualize.
Pin this workbook or only its chart to the Azure Dashboard.

Query SQL database using a Azure Logicapp

Sun, 05 Apr 2020 00:00:00 GMT

My goal was to define a LogicApp what can be called via HTTP request, query a SQL database with a select count query and return the result as text on the HTTP response body. This is how my workflow looks like:

First of all define a HTTP trigger:

The URL is generated after you save the step for the first time. By default HTTP triggers listen on HTTP POST verbs. You can change this to GET by adding a parameter on the trigger step.

Next we need to execute a SQL query like this:

Next we need to extract the value from the query result. This is a little complicate because the result of the SQL query step is dynamic and therefore the designer can not assist us.

Here is the expression I used:

body('Query_Event_Count')?['resultsets']['Table1'][0]['']

Notes:

Query_Event_Count is the name of the query step above.
The last part '' is the field name to extract. My select count() return a empty field.

Finally we need to set a HTTP response which is straight forward. Note that I just return plain text as the body. You can get fancy here.

Note on costs:

A Azure LogicApp costs per execution. Not much but it costs. If you plan to do a lot of HTTP calls this PoC is probably not what you are looking for.

Here is the complete source-code of my LogicApp:

{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Initialize_variable_2": {
                "inputs": {
                    "variables": [
                        {
                            "name": "countEvents",
                            "type": "integer",
                            "value": "@body('Query_Event_Count')?['resultsets']['Table1'][0]['']\n"
                        }
                    ]
                },
                "runAfter": {
                    "Query_Event_Count": [
                        "Succeeded"
                    ]
                },
                "type": "InitializeVariable"
            },
            "Query_Event_Count": {
                "inputs": {
                    "body": {
                        "query": "select count(Id) from Events"
                    },
                    "host": {
                        "connection": {
                            "name": "@parameters('$connections')['sql']['connectionId']"
                        }
                    },
                    "method": "post",
                    "path": "/v2/datasets/@{encodeURIComponent(encodeURIComponent('default'))},@{encodeURIComponent(encodeURIComponent('default'))}/query/sql"
                },
                "runAfter": {},
                "runtimeConfiguration": {
                    "staticResult": {
                        "name": "Query_Event_Count0",
                        "staticResultOptions": "Disabled"
                    }
                },
                "type": "ApiConnection"
            },
            "Response": {
                "inputs": {
                    "body": "@{variables('countEvents')} Events",
                    "statusCode": 200
                },
                "kind": "Http",
                "runAfter": {
                    "Initialize_variable_2": [
                        "Succeeded"
                    ]
                },
                "type": "Response"
            }
        },
        "contentVersion": "1.0.0.0",
        "outputs": {},
        "parameters": {
            "$connections": {
                "defaultValue": {},
                "type": "Object"
            }
        },
        "staticResults": {
            "Query_Event_Count0": {
                "outputs": {
                    "headers": {
                        "ResultSets": "Table1"
                    },
                    "statusCode": "OK"
                },
                "status": "Succeeded"
            }
        },
        "triggers": {
            "manual": {
                "inputs": {
                    "method": "GET",
                    "schema": {}
                },
                "kind": "Http",
                "type": "Request"
            }
        }
    },
    "parameters": {
        "$connections": {
            "value": {
                "sql": {
                    "connectionId": "/subscriptions/7be8.....",
                    "connectionName": "sql-2",
                    "id": "....."
                }
            }
        }
    }
}

Hope this helps others!

Show results of SQL query on Azure dashboard

Sun, 05 Apr 2020 00:00:00 GMT

Note: This post originally contained 2 images that could not be recovered during the WordPress migration.

I like to show the result of SQL Query on a Azure Dashboard. “Should be checken food” I thought. I checked a ton of options and finally got solution working.

Solution

One way I found was by writing a custom HTTP endpoint which returns Markdown text. I then added the Markdown widget and entered the endpoints URL as the external Markdown source.

This is how it looks like when my endpoint returned the string “58 Events”:

If you already have a AppService app or something similar you maybe like to integrate such “monitoring endpoints” in your solution. I didn’t like to to do it.

So how did implement the custom HTTP endpoint? Because I had issues with Azure Functions and query a SQL database in the past I’ve chosen Azure LogicApp to do it.

A LogicApp can be triggered on HTTP requests, can do SQL queries and can respond to HTTP requests. The down-side: LogicApps cost per execution. Not much but they cost. So its more proof-of-concept type of a solution. Read about my logic-app here…

Possible improvements:

Use Azure Functions instead of the LogicApp.
Use a web app instead of the LogicApp.
Log it as log-events and use Azure Monitor Workbook to query it and build a nice report (workbooks can be pinned to Azure Dashboard).

Azure DevOps: Retrieve Application Insights instrumentation key

Sat, 18 Jan 2020 00:00:00 GMT

Recently I had to set up Azure Application Insights instances via an Azure DevOps CI/CD pipeline using ARM templates. It was straightforward — except for retrieving the newly created instrumentation key.

The Problem

I needed the instrumentation key of the newly created AppInsights instance to store it in the application’s Azure Key Vault, where the application reads its secrets from.

I added an “Azure PowerShell” step to my pipeline to run a script with the right Azure context (subscription and resource group).

According to Microsoft’s documentation, there’s a PowerShell cmdlet called Get-AzureRmApplicationInsights that should do the job. However, this cmdlet returned no value — and no error either.

The Solution

I used three pipeline variables:

rg_name — the Azure Resource Group name
appinsights_name — the resource name of the AppInsights instance
appInsights_instrumentationkey — receives the instrumentation key

The following Azure PowerShell script did the trick:

Write-Host "Resource Group Name: $(rg_name)"
Write-Host "AppInsights Name: $(appinsights_name)"

$instrumentationKey = (Get-AzApplicationInsights -ResourceGroupName "$(rg_name)" -Name "$(appinsights_name)").InstrumentationKey

Write-Host "Instrumentation Key: $instrumentationKey"

Write-Host "##vso[task.setvariable variable=appInsights_instrumentationkey;]$instrumentationKey"

What’s Different

The key insight: use Get-AzApplicationInsights (the newer Az module) instead of Get-AzureRmApplicationInsights (the deprecated AzureRM module). Call it to get all properties, then access .InstrumentationKey.

The last line writes the value back to the pipeline variable appInsights_instrumentationkey using the ##vso logging command.

Putting It Together

A later step in the pipeline writes appInsights_instrumentationkey to a Key Vault, where the application picks it up at runtime.

Hope this helps!

Inject Xamarin Forms view models via IoC container

Sat, 18 Jan 2020 00:00:00 GMT

I started a new little project in my spare time which is a native mobile app for iOS and Android. Because the goal is to have as much shared code between iOS and Android as possible and still have native apps experience I choose the Xamarin Forms stack. Its also an excellent opportunity to refresh my XAML know-how.

The problem

I quickly got a screen working but missed a full fledged IOC container with proper constructor injection. There are libraries and article but most of them are outdated or even discontinued.

The Dependency Injection documentation got me half way started. It contains mostly general information about dependency injection, inversion of control and so on. But it also has a sample implementation in this GitHub repo which was very helpful.

For Xamarin Forms the recommended pattern for non-trivial apps is the MVVM pattern. Means we do a view using XAML (or C#) and connect it with a view-model using data-binding. The view-model contains all the UI logic and connect to other service classes – for example to retrieve data from a server.

What I like to do is having view-models instantiated automatically using a IOC container so all the constructor injection takes place. As the view-models are the central part of MVVM apps we get IoC for wast majority of the app just with this alone.

How to integrate an IOC container so the views get the view models injected and all dependencies are resolved as part of the constructor injection?

Solution

As I like SimpleInjector I choose this one but you can do basically the same with every other container that runs on .net Core and the Xamarin stack.

The basic concept as described in Microsoft’s documentation goes like this (took me a while to get this):

Create a ViewModelLocator which creates an attached property for the views called AutoWireViewModel (boolean).
When the AutoWireViewModel is set the ViewModelLocator looks for a matching view model class using a naming convention.
The view model gets resolved using SimpleInjector IoC container.
The view model instance then gets set as the views BindingContext.
The normal binding kicks in because BindingContext was set.

The View Model Locator

Here is my ViewModelLocator similar to the one from the Microsoft documentation. You see that the ViewModelLocator is a static class and hold a singleton reference to the SimpleInjector container. It also defines the AutoWireViewModel property. In the change handler OnAutoWireViewModelChanged you find the resolving of the view model and its assignment to BindingContext of the view. Change this logic to your needs.

In the static constructor I instantiate the SimpleInjector container called Container. I then call my helper IoCRegistrations.RegisterDependencies to do all the registrations.

public static class ViewModelLocator
    {
        private static readonly Container Container;

        static ViewModelLocator()
        {
            Container = new Container();
            IoCRegistrations.RegisterDependencies(Container);
        }

        public static readonly BindableProperty AutoWireViewModelProperty =
            BindableProperty.CreateAttached(
                "AutoWireViewModel",
                typeof(bool),
                typeof(ViewModelLocator),
                default(bool),
                propertyChanged: OnAutoWireViewModelChanged);

        public static bool GetAutoWireViewModel(BindableObject bindable)
            => (bool)bindable.GetValue(AutoWireViewModelProperty);

        public static void SetAutoWireViewModel(BindableObject bindable, bool value)
            => bindable.SetValue(AutoWireViewModelProperty, value);

        public static void RegisterSingleton<TInterface, T>() where TInterface : class where T : class, TInterface
            => Container.RegisterSingleton<TInterface, T>();

        public static T Resolve<T>() where T : class
            => Container.GetInstance<T>();

        private static void OnAutoWireViewModelChanged(BindableObject bindable, object oldValue, object newValue)
        {
            var view = bindable as Element;

            var viewType = view?.GetType();
            if (viewType?.FullName == null)
            {
                return;
            }

            var viewName = viewType.FullName.Replace(".Views.", ".ViewModels.");
            var viewAssemblyName = viewType.GetTypeInfo().Assembly.FullName;
            var viewModelName = string.Format(CultureInfo.InvariantCulture, "{0}ViewModel, {1}", viewName, viewAssemblyName);

            var viewModelType = Type.GetType(viewModelName);
            if (viewModelType == null)
            {
                return;
            }

            var viewModel = Container.GetInstance(viewModelType);
            view.BindingContext = viewModel;
        }
    }

This was the most tricky part.

Just for the record here the helper for the IoC registrations:

public static class IoCRegistrations
    {
        public static void RegisterDependencies(Container container)
        {
            container.RegisterSingleton<IDataStore<Item>, MockDataStore>();
            container.RegisterSingleton<ICountryDataStore, CountryDataStoreMock>();
            ...
        }
    }

Do whatever you need to do to get your stuff registered with the container.

Usage of the View Model Locator in the View

All that is needed now is that the view sets the AutoWireViewModel to true. Check out the line base:ViewModelLocator.AutoWireViewModel="true".

<?xml version="1.0" encoding="utf-8"?>
<ContentPage
    xmlns="http://xamarin.com/schemas/2014/forms"
    xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
    xmlns:views="clr-namespace:DiveSpots.Views"
    xmlns:base="clr-namespace:DiveSpots.ViewModels.Base;assembly=DiveSpots"
    base:ViewModelLocator.AutoWireViewModel="true"
    x:Class="DiveSpots.Views.WatersOverviewPage"
    Visual="Material"
    Title="{Binding Title}">

    <StackLayout>
        ...

Naming convention

I use the naming convention that the views are in the namespace DiveSpots.Core.Views and their name ends with ‘View’ while the view models are in DiveSpots.Core.ViewModels and there name is equal to the view but end with ‘ViewModel’. This can be changed or enhanced in OnAutoWireViewModelChanged.

SimpleInjector

To use simple injector with Xamarin Forms the only thing I had to do was to add the SimpleInjector NuGet package to the project.

Hope this helps.

Ensure public API surface using ApprovalTests

Sat, 16 Nov 2019 00:00:00 GMT

One of the things that made me think for some time is “How can I ensure my public REST-API does not change by accident?”.

Why

With todays IDE’s we have very powerfull automatic refactorings which is cool. The downside is that we easily can refactor a large pile of code what can affect the public API’s without the developer notice it.

When my public API’s (for example a REST-API’s my backend offers to partner companies) changes without notice it will be found very late the software development lifecycle. Worst case: after the release. This will result in a lot of work (and cost) for everybody involved.

The idea

So “Where is the problem?” you may ask: “Just write a bunch of unit-tests!”. The problem with this is that with the powerfull refactorings we usualy also refactor the unit-test code in one go. So the unit-test never becomes red even if we changed the public API. Therefore plain unit-tests are useless for this kind of tests.

We need something that does not get refactored automatically when we use automatic refactorings.

What I came up with is using ApprovalTests to dump the relevant data a in a approved-file (aka expectation file). This becomes our exectation of the unit-test result. The approval files don’t get changed by refactorings.

When they change I as a developer have to approve the new content manually (with the help of my diff tool) and therefore get noticed by my unit-test suite that something in the API changed.

How

ApprovalTests implementations are availlable for all major platoforms. For .Net you can install this Nuget package.

To test may REST-API’s I basically call action methods of my WebAPI controllers, serialize the output as Json and verify it using Approvals.

[UseReporter(typeof(DiffReporter))]
public class PersonControllerTests
{
    private readonly IPersonRepository personRepo = A.Fake<IPersonRepository>();
    private readonly PersonController controller;

    public PersonControllerTests()
    {
        controller = new PersonController(personRepo);
    }

    [Fact]
    public async Task GetDetails_Success_MustReturnObject()
    {
        // Arrange
        var personId = new Guid("51bd89eb-5ae6-4cfe-954a-f3f6a51352ab");

        // Act
        var result = await controller.GetDetails(personId)

        // Assert
        Approvals.VerifyJson(JsonConvert.SerializeObject(result));
    }
}

To enhace this one can also store the verb’s and URL’s. To do so you can wrap the above result in an anonymous object where you store additional information and serialize all of that into Json and approve it. It would be recommeded to implement a helper for doing this.

Alternative idea:

For REST-API’s an alternative idea is to generete the Swagger.json file you normaly need anyway and store it in your repo. Write a unit-tests that compares a freshly generated Swagger.json to the existing one. This would need some more advanced testing code / helpers but result in verify every aspect of the public API (Url’s, HTTP-Verbs, data-structures, supported headers, query-parameters, etc.)

Mapping with Dictionary instead switch

Wed, 30 Oct 2019 00:00:00 GMT

Recently I read the book “Refactoring: Improving the Design of Existing Code (2nd Edition)” by Martin Fowler (here…). A great book I highly recommend. Besides a lot of other useful informations and tips I started adapting to not use switch statements for mapping-logic.

Why not using Switch?

The point is that a switch statement can do a lot more things then just mapping value. From my own experience I can confirm that regularly a switch’s case branch does more then one thing. More then just map value A to B. This leads in side-effects of the mapping logic. Further more the code does not clearly show its intent anymore, is harder to read, refactor and extend.

So we look for a better solution for mapping values. A solution that does only map values from A to B and no other logic can be placed into. It should show clearly that it only does a simple value-mapping.

That’s where dictionaries come into the play.

Dictionaries

Dictionary is a collection data-stucture that host key-value items. One can access values very fast by its indexer; the key. They have an optimized and short syntax to do exactly this. If you don’t use delegates as the value you can not add logic. Its a simple value translation table. So the intent should be very clear.

Example

Let’s map an enum value to a string (can be anything else). Here is the enum:

public enum Gender
{
    Female,
    Male,
    Other
}

Map value using a switch statement:

string MapGenderToString(Gender gender)
{
    swtich (gender)
    {
        case Gender.Female:
            return "Woman";
        case Gender.Male:
            return "Man";
        case Gender.Other:
            return "Other";
        default:
            throw new ArggumentRangeException();
    }
}

The problem here is that you can easily add any kind of logic (code) within one or more case blocks. This would be a code-smell as written before.

Map value using a dictionary:

string MapGenderToString(Gender gender)
{
    var mapping = new Dictionary<Gender, string>
    {
        { Gender.Female, "Woman" },
        { Gender.Male, "Man" },
        { Gender.Other, "Other" }
    }

    return mapping[gender];
}

It’s way harder – but not impossible – to missuse this mapping code.

Bonus value: Because the mapping is so simple one whould not even need an extra method for this. The mapping could be defined as a readonly-field and used directly in the parent code.

class PersonMapper
{
     private readonly genderMapping = new Dictionary<Gender, string>
    {
        { Gender.Female, "Woman" },
        { Gender.Male, "Man" },
        { Gender.Other, "Other" }
    }

    public PersonDto MapPerson(Person person) =>
        new PersonDto
        {
            ...
            Gender: genderMapping[person.Gender],
            ...
        }

}

Hope this gives some inspiration. Let me know what you think about this.

Chapter 1: My journey in Clean Architecture and Domain Driven Design

Sun, 22 Sep 2019 00:00:00 GMT

Last year I’ve read several new books about software development. One of them was the book “Clean Architecture: A Craftsman’s Guide to Software Structure and Design” by Robert C. Martin (aka Uncle Bob).

On the other hand I started to use more Domain Driven Design (DDD) practice on my day-job. I read about DDD already several times in the past but never got it really started.

On one hand I like to practice DDD but even more I was hungry to do a Clean Architecture journey and give it a try. What I needed was a peace of software to get started with. Lucky my I had a web application that needed some love anyway. Voila: my journey was ready to start!

In this multipart blog post I write what challenges I faced and what solutions I finally came up with.

Chapters:

Chapter 1: (this one) Intro and setting the stage
Chapter 2: Clean Architecture and DDD
Chapter 3: Controllers
Chapter 4: Use-Cases & Interactors
Chapter 5: Entities & Domain Events
Chapter 6: Presenters & View-Models
Chapter 7: Drivers & Data Access
Chapter 8: Conclusion and final words

The web application

The web application I liked to modernize is the one that run on https://tauchbolde.ch. It’s the website of my diving group. Beside some static content and some other diving calculation-gadgets it has two main domains (aka modules): Diving events(Events in short) and Logbook.

The Events domain allows divers to create upcoming diving events/dives and register for them incl. building buddy-teams, place comments, download ICal files, etc. The Logbook domain allows members to write articles about past events with image upload etc. Kind of a simple blog.

The samples in this journey are from the Logbook domain as this simple blog is way easier to understand than then diving stuff.

The website is build using .Net Core 2.2, ASP.Net MVC Core 2.2 and Entity Framework Core 2.2. All the soure-code can be found here on GitHub. The Azure DevOps pipelines are here.

Problems with the old architecture

The old application was build like many common MVC application are: MVC-Controller uses repositories to load/save data and builds the view-model ad-hoc in the controller and triggers the view.

As the logic parts had grown I moved them into what was called domain services and therefore removed data-access from the MVC controllers.

With the time some domain-services started to re-use other domain-services and cross-cutting general purpose services. If became less clear what depends on what and unit-testing these domain services became a challenge on its own.

Read on about Clean Architecture & Domain Driven Design here…

Chapter 2: Clean Architecture & Domain Driven Design

Sun, 22 Sep 2019 00:00:00 GMT

Clean Architecture

As I already stated Clean Architecture was the main reason I started this journey. If you are seriously interested in reading more about Clean Architecture I kindly recommend Uncle Bob’s book. The book features several parts. One introduce the SOLID principals on architecture level while others focus more on the Clean Architecture itself. The “problem” with the book is that does not have easy adoptable guidelines. There is some sample code but not in the form for general re-usage. So it leaves some room for interpretation – which I did.

On this blogpost Uncle Bob summarized the core ideas behind the Clean Architecture (CA). Before continue here please read it first.

Copyright by Rober C. Martin

If you look at the above diagram from Uncle Bob you’ll see two things.

The “rings” or “layers”. An inner ring does not know anything about outer rings and outer rings only know the next inner ring. In the center there are the business entities. Uncle Bob leaves it open if you do Domain Aggregates or some other kind of business-objects. Then there are the use-cases from the business. This is where the “dance of the entities” is orchestrated. In other architecture this is known as the application-layer.

When you look at the bottom right there is a second diagram: The flow of control. At first I didn’t get this one while later I spent serious time thinking on how to implement something comes close to this flow in a MVC web application. More on this later.

The nice thing about CA (and others like The Onion Architecture) is that it nicely separates concerns and therefore dependencies. This leads to way easy writing of automated unit-tests which is a big deal from my point of view.

I somewhere read that CA can nicely be combined with Domain Driven Design – it can act like a host to DDD so let’s quickly address DDD.

Domain Driven Design (DDD)

If you like to get quick intro to the core of DDD I can recommend the book DDD Quickly by InfoQ freely available online. Another good but a little thicker book is Domain Driven Design Distilled by Vaughn Vernon.

How I see the big value of DDD:

The DDD principals puts the business-perspective back on the code. The same does CA. The goal is that we can see the main business values our app addresses in quickly in our code without the need of digging to tons of techie stuff first. It should be obvious for a reader what main business-needs the software addresses. Its why the software has the right to exist. Uncle Bob calls this “[Screaming Architecture]”(https://blog.cleancoder.com/uncle-bob/2011/09/30/Screaming-Architecture.html).

One of the nice things with DDD is that one build the core of the code around the business needs in a nice object-oriented manner.

With these basics in place let’s jump into my implementation …

Chapter 3: Controllers

Sun, 22 Sep 2019 00:00:00 GMT

In CA a controller takes input, validates it and convert it to the format best used for the use-cases layer (especially the interactors).

This is where I did the compromises and went away from Clean Architecture. In my application the controllers are MVC-Controllers. They do the input-to-usecase-thing but also have to deliver the final IActionResult (kind of the HTTP-Response). If I like to do a really clean architecture I need separate the web-stuff and the non-web-stuff for example by introducing a Use-Case-Controller (called by the MVC-Controller). I decided that this is so far will remain a web-application for the foreseeable future so I made the compromise to use the MVC-Controller as my controller.

A simple controller-action in my application looks like this:

[HttpGet]
[Authorize(Policy = PolicyNames.RequireTauchboldeOrAdmin)]
public async Task Publish(Guid id)
{
    var publishLogbookEntry = new PublishLogbookEntry(id);
    var result = await mediator.Send(publishLogbookEntry);
    if (!result)
    {
        ShowErrorMessage("Fehler beim Publizieren des Logbucheintrages!");
    }

    ShowSuccessMessage("Logbucheintrag erfolgreich publiziert.");
    return RedirectToAction("Detail", "Logbook", new {id});
}

Note that this is a controller-action with no further output besides a success message so I don’t use presenters etc. More on presenters and view-models later on.

What the above code does is create a use-case command object of type PublishLogbookEntry which holds the data used for this simple use-case to publishing an article. In this case its just the ID of the LogbookEntry to publish.

public class PublishLogbookEntry : IRequest<bool>
{
    public PublishLogbookEntry(Guid logbookEntryId)
    {
        LogbookEntryId = logbookEntryId;
    }

    public Guid LogbookEntryId { get; }
}

Notes:

The class implements the IRequest interface of MediatR. MediatR is a in-process message-queue. It allows me to send requests to one recipients or broadcast notifications to multiple subscribers. I use it to decouple things further more. In this example the controller-action creates a PublishLogbookEntry request and sends it using MediatR. The controller has not even a clue that there will be a use-case interactor or even a domain-entity. He just sends this request and expect a response from someone.
The implementation of ÌRequest means that this is a “request” object that gets sent to one receiver and returns a bool result to the caller.
I try to implement as many classes as immutable classes as possible. This is why there is no setter on the LogbookEntryId and it can only be set using the constructor and never be changed afterwards. For the “why immutable” you may read this blog from Eric Lippert on Immutability in C#. There will be other places in the code where I use newer C# language features to enforce immutability.
In addition to the use-case command object I often also implement a validator (see Fluent Validations) for the command. These command validators get picked up by MediatR automatically (if configured correctly), validate the request object and return validation errors before any further processing of the command is done. These things are called pipeline behavior.

Let’s go one to one of the most important peaces of my implementation, the Use-Cases & Interactors …

Chapter 4: Use Cases & Interactors

Sun, 22 Sep 2019 00:00:00 GMT

Note: This post originally contained 1 image that could not be recovered during the WordPress migration.

In my architecture the use-case command objects get served by so called Use-Case Interactors. Technically these are MediatR ÌRequestHandler which handle the use-case commands. Interactors orchestrate “the dance of the entities” as Uncle Bob says. This means that they do a mix of loading and saving entities from/to the persistence layer, call business logic using the entitiy-methods, calling other services, connect to external systems etc. They are orchestrators or moderators if you wish. Its one of the main parts for the application layer.

I you look at solution explorer you immediately see what use-cases this application addresses. This way even the business and domain-exports understand the scope of the application. This is one of the really cool things.

So let’s jump to a sample code of the use-case interactor which publishes a LogbookEntry for public viewing:

[UsedImplicitly]
internal class PublishLogbookEntryInteractor : IRequestHandler<PublishLogbookEntry, bool>
{
    [NotNull] private readonly ILogger<PublishLogbookEntryInteractor> logger;
    [NotNull] private readonly ILogbookEntryRepository dataAccess;

    public PublishLogbookEntryInteractor(
        [NotNull] ILogger<PublishLogbookEntryInteractor> logger,
        [NotNull] ILogbookEntryRepository dataAccess)
    {
        this.dataAccess = dataAccess ?? throw new ArgumentNullException(nameof(dataAccess));
        this.logger = logger ?? throw new ArgumentNullException(nameof(logger));
    }

    public async Task<bool> Handle([NotNull] PublishLogbookEntry request, CancellationToken cancellationToken)
    {
        if (request == null) throw new ArgumentNullException(nameof(request));

        logger.LogInformation("Publish Logbook-Entry {logbookEntryId}", request.LogbookEntryId);

        var logbookEntry = await dataAccess.FindByIdAsync(request.LogbookEntryId);
        if (logbookEntry == null)
        {
            logger.LogError("Logbook-Entry {logbookEntryId} not found", request.LogbookEntryId);
            return false;
        }

        try
        {
            logbookEntry.Publish();
            await dataAccess.UpdateAsync(logbookEntry);

            logger.LogInformation("Logbook-Entry {logbookEntryId} published successfull.", request.LogbookEntryId);
            return true;
        }
        catch (Exception ex)
        {
            logger.LogError(ex, "Error publishing logbook entry {logbookEntryId} {logbookEntryTitle}", logbookEntry.Id, logbookEntry.Title);
            return false;
        }
    }
}

Notes:

The interactor implements ÌRequestHandler so MediatR use them to handle PublishLogbookEntry request that return a bool result.
Use private readonly fields that can be set only within the constructor of the class and therefore are immutable.
The constructor parameters normally get injected by IoC container but in the unit-tests fakes are provided for them (see Dependency Inversion principal).
The constructor implementation uses C# 7 throw expressions to check if the value is not null. I also use R# annotations like [NotNull[] and [UsedImplicitly]. These annotations help R# to help you by hinting where you might get a null-reference exception if you do not check for null.
Another thing I got from Clean Architecture is to make as much as possible accessible only internal. Everything that I declare public is considered an external API that needs to be supported and therefore can not easily change. When everything in a assembly is public its a sign of a code-smell. Regarding Uncle Bob hiding things by not making them public is the biggest advantage for object-oriented programming (OOP).

Now that we have seen the application layer – or Application Business Rules as it is called CA – its time to go to the Enterprise Business Rules. Let’s look at the Entities and Domain Events here …

Chapter 5: Entities, Domain Events and Policies

Sun, 22 Sep 2019 00:00:00 GMT

Entities

Most interactors call one or more methods on the entities. These entities are the “business objects” and the methods implement the “business logic”. In a event-sourced system the entities are known as domain-aggregates.

An entities method should only do things on the local entity. It must not depend on other entities (aka aggregates). It may use other services that get pass into the method using a parameter but most if the time this is a sign for a code-smell. For such tasks we will use domain-events and policies (see below).

In the above sample the following line is the meat:

logbookEntry.Publish();

The LogbookEntry entity looks as following.

public class LogbookEntry : EntityBase
{
    ...

    public bool IsPublished { get; internal set; }

    ...

    public void Publish()
    {
        if (IsPublished) return;

        IsPublished = true;
        RaiseDomainEvent(new LogbookEntryPublishedEvent(Id));
    }

    ...
}

See full source-code.

Notes:

The Publish() method first checks if the LogbookEntry is already published because then nothing needs to be done anymore. If not the IsPublished flag will be set and a domain event LogbookEntryPublishedEvent will be raised to indicate what just happened to other parts of the application (more on this later).
The IsPublished as an internal setter. This is to simplify writing unit-tests. My unit-test assembly is listed as a so called friendly assembly to the domain assembly so internals are visible to unit-tests too.
The EntityBase base-class features the `Ìd“ property as well as the storage of the uncommitted domain-events. See here for soure-code.

Domain Events

A domain event (see DDD) indicates that something important happened on a entity (aka domain aggregate). They are where the open/close principals gets into the play. Some call them extensibility points. It’s the way entities can communicate with the outer world.

The domain event LogbookEntryPublishEvent gets broadcasted using MediatR after the data-access did persist the entitiy. Because they are broadcasted to the in-process messaging zero to many subscribers can consume them and do various others things. Normally these things are not so closely related to the previous domain entity logic or need to interact with more then just the one current entity.

In our example the domain event LogbookEntryPublishedEventlooks like this:

public class LogbookEntryPublishedEvent : DomainEventBase
{
    public Guid LogbookEntryId { get; }

    public LogbookEntryPublishedEvent(Guid logbookEntryId)
    {
        LogbookEntryId = logbookEntryId;
    }
}

As you may have noted I have a baseclass for my domain events call DomainEventBase. This just inherit from INotification records the time when the event happened:

public abstract class DomainEventBase : INotification
{
    public DateTime DateOccurred { get; protected set; } = DateTime.UtcNow;
}

In a event-sourced application the donain-event would then be stored in the event-store and projectors will listen on then to to update their read-stores (eg. SQL tables). In my application I do not do event-sourcing so the domain-events serve only for notifying other building blocks that can do further processing. Policies are such building building blocks.

After we have the interactors in place which do the “dance of the entities” as well as having the entities with the business logic and the domain-event it’s time to move on what happens after the domain-events are published.

Policies

Policies are application-logic classes (in the application layer) that listen to domain-events. Policy then can do additional work that is not directly part to the previous domain method but to its result (the domain-event). The do work with other services or entities/aggregates outside of the initial entity/aggregate. Entities/aggregates should depend on other entities/aggregates.

To do their stuff Policies can access all of the application- and domain-level infrastructure – just like interactor can. This means they can load and update domain entities using repository-interfaces, call domain entity methods to execute other business-logic or they can use other application level services.

In our sample for publishing logbook entries we have two policies in place that listen to the LogbookEntryPublishedEvent domain-event.

The first one is LogTelemetryLogbookEntryPublishedPolicy which does only one little thing: it uses the ITelemetryService application-service to record the publish use-case as a telemetry-event. In my case they get submitted to Azure AppInsights for further logging and analytics.

The second policy is a little bit more interesting: PublishNewLogbookEntryNotificationPolicy. The job of the policy is to publish Notification‘. Notifications – in my domain – means a notification to users about something that happened. These notifications get picked up later on and an email newsletter is generated and sent every user.

Let’s take a look at this policy:

[UsedImplicitly]
public class PublishNewLogbookEntryNotificationPolicy : INotificationHandler<LogbookEntryPublishedEvent>
{
    [NotNull] private readonly IDiverRepository diverRepository;
    [NotNull] private readonly ILogbookEntryRepository logbookEntryRepository;
    [NotNull] private readonly INotificationPublisher notificationPublisher;

    public PublishNewLogbookEntryNotificationPolicy(
        [NotNull] IDiverRepository diverRepository,
        [NotNull] ILogbookEntryRepository logbookEntryRepository,
        [NotNull] INotificationPublisher notificationPublisher)
    {
        this.diverRepository = diverRepository ?? throw new ArgumentNullException(nameof(diverRepository));
        this.logbookEntryRepository = logbookEntryRepository ?? throw new ArgumentNullException(nameof(logbookEntryRepository));
        this.notificationPublisher = notificationPublisher ?? throw new ArgumentNullException(nameof(notificationPublisher));
    }

    public async Task Handle([NotNull] LogbookEntryPublishedEvent notification, CancellationToken cancellationToken)
    {
        if (notification == null) throw new ArgumentNullException(nameof(notification));

        var recipients = await diverRepository.GetAllTauchboldeUsersAsync();
        var logbookEntry = await logbookEntryRepository.FindByIdAsync(notification.LogbookEntryId);
        var author = await diverRepository.FindByIdAsync(logbookEntry.OriginalAuthorId);
        var message = $"Neuer Logbucheintrag '{logbookEntry.Title}' von {author.Realname}.";

        await notificationPublisher.PublishAsync(
            NotificationType.NewLogbookEntry,
            message,
            recipients,
            relatedLogbookEntryId: notification.LogbookEntryId);
    }
}

Notes:

The policy uses the IDiverRepository and ILogbookEntryRepository for accessing the entities it needs.
The policy gets all the data it needs in the notification parameter which is the domain-event. In this sample it gets the notification.LogbookEntryId which it uses to load details about the LogbookEntry from the database.
The INotificationPublisher.PublishAsync() is used to publish the notification to all interested users. Note that INotificationPublisher is general application-service which gets injected into the policy as a constructor parameter.

After we set up the basic stuff its time to move to the most tricky part to fiddle out how to do it: Presenters and View-Models.

Chapter 6: Presenters & View models

Sun, 22 Sep 2019 00:00:00 GMT

So far we covered the more command/write-side focused use-cases. It was pretty straight forward to implement this in a Clean Architecture fashion. What really got me think for a longer time (days or weeks) was the kind of use-cases where real user-output (eg. HTML) must be produced. This is the part where the Presenters come into the play.

If we take a look again at the control-flow in Uncle Bobs diagram at the beginning we see the following:

Controller calls the interactor using the input-port.
Interactor calls the output-port.
Presenter implements the output-port and renders things to the UI.

This means that a presenter can render stuff on its own to “the user-interface”. This is not how standard MVC works. In standard MVC we have the following flow.

Controller gets the user-input.
Controller coordinates some kind of logic to do things.
Controller returns a action result like a view result with the view-model needed to render the view.
The ASP.Net pipelines fires the view-engine with the viewname and view-model from the view-result after the controller-action was finished.

Hmm… so how to bring these worlds closer to each other. I know that Uncle Bob would not like to use standard MVC at all because it highly couples our overall architecture and dependencies to a framework we don’t have control over it. Regarding Uncle Bub, even the ASP.Net MVC part of the application should be an implementation detail that does not affect your architecture.

I, on the other hand, have a existing MVC application and I like to use MVC as it was thought by its inventors for several reasons. First one is that if other developers join they at least know the basics. Other reason is documentation, communities and general help.

Maybe one day I find the time to have a closer look at frameworks like Nancy where I think I am more free with these architectural decisions.

So again, how can I get things closer to Clean Architecture?

I did some googling and found a few people thought about this already. Plainionist wrote a really nice series of article about this here. Another nice thread I found on here Stack-Overflow and one here on CodeReviews.

Regarding the control-flow we like to get this: controller-interactor-presenter.

Option 1: The controller gets the interactor result and create a view-model and returns the view. This will not result in the control flow as of Clean Architecture because no output-ports and presenters are used at all. The flow would be controller-interactor-controller.

Option 2: The controller gets the interactor result and forwards it to a presenter which builds and returns the view-model to the controller so the controller can return a view based on the presenters view-model. This is also not the Clean Architecture control-flow as its controller-interactor-controller-presenter-controller.

Option 3: The controller triggers the interactor and provide the presenter as the output port. The interactor then calls the presenter which builds the view-model. Finally the controller gets the view-model from the presenter and returns the view. Again its not exactly the Clean Architecture control-flow: controller-interactor-presenter-controller.

Option 4: Like option 3 but split controller into MVC- and Clean-Architecture-controller. This would result in the flow MVCcontroller-controller-interactor-presenter-MVCcontroller. Again not quite the Clean Architecture way. But if we remove the technical MVC-Controllers from the above equation we are in the green zone of Clean Architecture. I guess Uncle Bob would also like this because the controller-logic is split away from the MVC-Framework and therefore its dependency.

Why don’t we get the exact Clean Architecture flow? Its because we use MVC where we have to build a view and return the view to the framework which finally renders the HTML. This means we don’t have a self-containing render-flow. At least not if we stick to the MVC framework and therefore can not freely control the flow.

So what did I choose?

I like to go close with Clean Architecture but don’t do too much extra work. So I went with … tatarataaa … Option 3 (with one eye to option 4 – maybe one day I go for 4).

The next question regarding presenters was:

How to inject the presenter into the interactor?

I basically see two options here:

Option 1: Inject the output-port (eg. presenter) using a interactor constructor parameter (constructor injection).

Option 2: Pass the output-port (eg. presenter) to the input-port into the interactor.

The advantage of the first option is that you register the presenter as the output-port implementation with your DI and let the DI inject it. This way the controller does only know the presenters interface to get the view-model or the view out of it.

The advantage of option 2 is that one can pass different output-port implementations to the same interactor depending where the interactor is used. This increase the re-usability of the interactor.

What did I choose? I switched forth and back between these options but finally decided for option 2 because the very same application instance may not only host MVC- but also WebAPI controllers in the future. Using option 2 I can re-use the interactors for my REST-API and for my HTML-App. The only difference is that I pass in another output-port implementation. MVC-Presenter or JSON-Resenter.

With the stage set on how to implement Clean Architecture output-ports and presenters let’s have a look at the code I came up:

First the controller-action to renders the detail page of a LogbookEntry:

public async Task Detail(Guid id)
{
    var presenter = new MvcLogbookDetailsOutputPort(relativeUrlGenerator, logbookDetailsUrlGenerator);
    var interactorResult = await mediator.Send(new GetLogbookEntryDetails(id, presenter, await GetAllowEdit()));
    if (!interactorResult.IsSuccessful)
    {
        ShowErrorMessage("Fehler beim laden der Daten des Logbucheintrages!");
    }

    return View(presenter.GetViewModel());
}

Notes:

First the MVC-specific presenter is created including all the dependencies the presenter needs. I’m thinking of changing this so the presenter get created and injected into the controller using DI using constructor injection.
The presenter implements the interactors output-port.
The interactor-command (aka interactor input) is sent including a reference to the presenter as the output-port.
Basic error handling is done based on the interactor result.
Finally the controller asks the presenter for the view-model and returns a view. I am thinking if I should move the View() statement to the presenter so the presenter will not only return a view-model but an entire view. This would take further UI-logic away from the controller and move it to the presenter.

Let’s see how to presenter looks like:

public class MvcLogbookDetailsOutputPort : ILogbookDetailOutputPort
    {
        [NotNull] private readonly IRelativeUrlGenerator relativeUrlGenerator;
        [NotNull] private readonly ILogbookDetailsUrlGenerator detailsUrlGenerator;
        private LogbookDetailViewModel viewModel;

        public MvcLogbookDetailsOutputPort(
            [NotNull] IRelativeUrlGenerator relativeUrlGenerator,
            [NotNull] ILogbookDetailsUrlGenerator detailsUrlGenerator)
        {
            this.relativeUrlGenerator = relativeUrlGenerator ?? throw new ArgumentNullException(nameof(this.relativeUrlGenerator));
            this.detailsUrlGenerator = detailsUrlGenerator ?? throw new ArgumentNullException(nameof(detailsUrlGenerator));
        }

        public void Output([NotNull] GetLogbookEntryDetailOutput interactorOutput)
        {
            if (interactorOutput == null) throw new ArgumentNullException(nameof(interactorOutput));

            viewModel = new LogbookDetailViewModel(
                interactorOutput.AllowEdit,
                interactorOutput.LogbookEntryId,
                interactorOutput.Title,
                interactorOutput.IsFavorite,
                interactorOutput.CreatedAt.ToStringSwissDateTime(),
                ...)
            );
        }

        public LogbookDetailViewModel GetViewModel() => viewModel;
    }

Notes:

The presenter implements the ILogbookDetailOutputPort by implementing the Output() method with the signature as defined by the output-port. Note that the interactors only know and depend on the output-port interface. He doesn’t know who implements it nor that there are presenters or even view-models at all.
The view-model can be retrieved from the presenter using get GetViewModel() method.
The presenter converts the output from the interactor to a view-model so view view has an easy job. This means that the presenter converts complex objects to with basic data-types. In the above example you see that DateTime types get converted into strings so the views don’t need to do such conversions. More complex views get an more specialized view-model so the presenter for example builds multiple lists based on one interactor output. As said: the view should have as less logic as possible. Address render-logic in the presenter if possible.

Let’s continue here …

Chapter 7: Drivers & Data Access

Sun, 22 Sep 2019 00:00:00 GMT

Drivers

Regarding Robert C. Martin, technical subsystems like storage, user-interface, external systems etc. are an implementation detail what should not bleed into your application logic. Put them behind a nice interface and implement them in the most outer ring: Framework & Drivers.

To do so my project has the Drivers solution folder. By the time of writing you find my implementations of database access (SQL with EF Core), image processing, email delivery and blob storage in here.

Each of these assemblies cover a topic by implementing their interfaces agains a concrete technology. If I like to use other technology to drive a topic in the future I simply replace these assemblies with the new implementations.

Data-Access

A few words on the data-access. I’ve implemented the SQL data-access using Entity Framework Core. I decided to not do another set of classes for the data-entities as they would be a 1:1 copy so far and I would need a ton of mapping code again.

Instead I put the mapping definitions of my entities into the data-access. I think Uncle Bob would not want this.

This may change when I go more into DDD style domain “aggregates” instead of relational entities. So far I did not have to need to change this.

I’m also thinking to move away from SQL to Azure Cosmos DB (MongoDB for local development). The main problem for this move is the ASP.Net Identity for which I used the standard EF Core implementation so far. I have to port this over from EF Core. Maybe Microsoft will provide a CosmosDB implementation one day.

The other issue with other database then SQL-Server is that for this project I have a very limited monthly budget and the SQL-Server is by way the cheapest option I found on Azure. $5/month while all other options I found where starting at $30/month.

One option I found which is extremely cheap (1-2 cents/GB/month). It first sound strange but Azure Storage. But if I move over to complete aggregates I could store them in JSON files/blobs. The only constrains would be complex queries but as I don’t have big-data this may be an option for persisting my data.

Let’s head over to my final thoughts and conclusions here …

Chapter 8: Conclusion and final words

Sun, 22 Sep 2019 00:00:00 GMT

I’ve spent a lot of time exploring these topics over the last months. I hacked in the subway every day and rack ones brain after kids where in bed. I’m quite happy with the result but I also see room for improvement here and there.

The nice thing about Clean Architecture (and others) is that they push you to do small separated peaces that do only one thing (separation of concerns) and to decouple thinks. Further more one does not only decouple but also organize them into layers the way that inner layers only know the general interface but not the concrete implementation.

This results in excellent test-ability and one can easily replace an implementation without changing the overall concept.

The nice thing about Domain Driven Design is that the business logic and flows are put into the center and that one builds a “domain model” out of it in a nice object-oriented fashion. We no longer have dumb objects that gets manipulated by services all over which are more procedural programming like.

The overall architecture I came up with in the sample application uses the same building blocks and clearly defined flows for all use-cases. I don’t have these strange service-dependencies all over.

I am also forced to think more about the business needs of a use-case. For example I need a use-case to start with. I need to think about how things are named in business terms (entities, entity-methods). This builds the ubiquitous between the business and the techs.

On the other hand this kind of architecture leads in more lines of code. But I think its worth it!

… at least if the application not only a trivial CRUD app but has business flows / logic.

Thanks for reading. Cheers!

Update SQL database using EF Core and Azure DevOps

Mon, 08 Jul 2019 00:00:00 GMT

If you write an app you probably need a way to store data. If you do it using a SQL Database (eg. SQL-Server) you need a way to change your Database-Schema sooner then later.

There are several options on how to do this in the Microsoft eco-system. Here I describe a very easy and simple way on how I do it for my personal project over at https://tauchbolde.ch.

Goal

First of all my app is a ASP.Net Core MVC 2.2 app using EF Core 2.2 with SQL-Server. More precisely with Azure SQL-Server. The application is hosted on Azure AppService. I use Azure DevOps to fully automate my build and release pipeline.

My goal was to automatically run the schema-migration as part of my automated deployment. The solution should be straight forward and transparent. It also should not need much of custom code.

Solution

I’m using EF Core’s Migrations to add migrations as soon as my schema needs to change. Normally I commit this migration together with my modified ORM model classes. Migrations are generated and added to the project using the following command:

dotnet ef migrations add

These migrations can be run on the command line using the EF CLI command:

dotnet ef database update

All pretty standard so far.

Instead of running the above commands when deploying and hope they will succeed I like to do it more SQL-like and more transparent. Therefore I maintain one single SQL-Script that updates any schema-version to the latest version.

Such an SQL-Script can easily be tested locally or on test-servers and with different datasets. If something goes wrong on production system’s migration we can fix the SQL-Script manually and are good without building a new software package.

Lucky me EF Core has this feature built into. I use a command line like this to generate the SQL-Script:

dotnet ef migrations script -i -o ../../sql/update_to_latest.sql

The generated SQL-Script contains the IF-Statements to check which migrations needs to be applied and which are already. It also takes care of updating the migration-history-table.

My build-job copies the sql/update_to_latest.sqlscript into the build artifact.

For my release-jobs I’ve created a Task Group called “Migate DB to latest”. I used the task group so I can re-use this step on all my environments (actually stage and prod). The Task Group contains one step of type Azure SQL Database deployment and executes the above SQL-Script.

Finally I’ve added this Task-Group to each environments release-steps.

Important: To get the connection to your SQL-Server working you have to add the connection in Azure Service Connections.

You can find my real-world docs over at GitHub.

Update Azure Pipeline BuildNumber using PowerShell

Mon, 21 Jan 2019 00:00:00 GMT

I recently moved a build-job for our front-end library over to Azure Pipelines. This build-job builds multiple TypeScript projects where each one produces its own NPM package using the Lerna mono-repo toolkit.

We manage the version of the packages using the Lerna CLI. Lerna updates all the projects package.json as well as a file called lerna.json in the root of the repo.

The goal for our release build-job was to update the BuildNumber of the job instance from within the running build-job to the version configured in lerna.json and suffix it with the original build-number (which is a increasing integer number). The build-job should be listed as 1.4.3.598 instead of #598.

Solution

I created a little PowerShell script which I checked into the repo and added it as one of the first build steps to the build-job.

$lernaPath = "$Env:BUILD_SOURCESDIRECTORY\lerna.json"
$json = Get-Content "$lernaPath" | Out-String | ConvertFrom-Json
$version = $json.version
$buildNumber = $Env:BUILD_BUILDNUMBER;

Write-Host "##vso[build.updatebuildnumber]$version.$buildNumber"

The first line gets the path of the lerna.json. The second line reads the file content and converts its JSON content into a PowerShell object. The fourth line gets the version number out of that object. The fifth line gets the original build-number from Azure Pipelines using the environment variable.

The most important line ist the last one. It writes a special text to the console which gets interpreted by Azure Pipelines. The ##vso[... is the trigger for Azure Pipelines. The function build.updatebuildnumber is the Azure Pipeline function that should be called. Finally the $version is the variable assigned in line three and $buildNumber the variable from line four. The both variables get concat with a ..

Fix sign-out with ASP.NET Identity Core 2.1

Fri, 16 Nov 2018 00:00:00 GMT

For a new app I’m writing using ASP.Net MVC Core 2.1, EF Core 2.1 and ASP.Net Identity 2.1 I had to implement the sign-out / log-out functionality. You may ask why, because this is shipped with ASP.Net Core OOTB? Yes – it is. But in the default template with ASP.Net Identity Core EF 2.1 it does not work.

I think I can work but depends on the kind auf authentication you activate. I use token authentication via cookies. The logoff action shipped with ASP.Net Core 2.1 doesn’t work for this. It forward you to the log-off page telling you, that you have been logged out but actually you are still logged in.

I figured that the cookie with the authentication token is still present in the the browser session – even after you called the default logout action. The token-authentication works the way the server generate an encrypted authentication token (basically a string) that is passed forth and back using a cookie. The server validates this token with each request. Normally there is no server-side token-store or so. So there is no need to actively log-out on the server-side.

To fix it I implemented my own action which removes this cookie and then redirect to the start-page of my app. To do this more reliable I gave the authentication cookie my own custom name.

In ConfigureServices() in Startup.cs I placed the following code to configure authentication:

services.ConfigureApplicationCookie(options =>
{
    // Cookie settings
    options.Cookie.HttpOnly = true;
    options.ExpireTimeSpan = TimeSpan.FromMinutes(30);
    options.LoginPath = "/Identity/Account/Login";
    options.AccessDeniedPath = "/Identity/Account/AccessDenied";
    options.SlidingExpiration = true;
    options.Cookie.Name = GlobalConstants.AuthCookieName;
});

Check the custom name (stored in a global constant) for options.Cookie.Name.

Then I wrote my little logout action like this:

[Authorize]
[HttpGet]
public IActionResult Logout()
{
    Response.Cookies.Delete(GlobalConstants.AuthCookieName);
    return RedirectToAction("Index");
}

Finally I placed a link to this new Logout action in my user-menu.

That’s it. Hope it helps.

No Estimates

Thu, 23 Aug 2018 00:00:00 GMT

Note: This post originally contained 5 images that could not be recovered during the WordPress migration.

I was at a meet-up from the Software Craftsmanship Zürich on the topic of “No estimates QA session” with Vasco Duarte. I read a few things about “No estimates” (aka “NE”) before on Twitter. I got questions on how this should or even can work in real-life. This article is a summary of some of the things I took away from that meet-up.

Estimates don’t work in large scale

The main reason for the whole #noestimates idea is the fact that in practice estimates do not work. If a team matches a deadline it’s normally not because of the precise estimates they did but because of smart human being. We as humans will cut corners and try to improvise so we can make the deadline.

Vasco showed an experiment that a team at Microsoft did. They set up a new project to develop some peace of software. The timeframe for this was about 10 months or so. They did the hard work and estimated the entire backlog using story-points just as we know from agile/scrum practices using values from Fibonacci (1, 2, 3, 5, 8, …). Then they did the same only using the values 1, 2 and 3. So scaling was different. Finally they set all stories to the same story-point value. The result was that all the forecasts showed a possible release-date within the same two or three weeks over the whole period of ten months.

The project-leader then came to the conclusion that they don’t need estimates. They are useless and the team can spend the time for more useful things instead like refining and slicing user-stories. Instead of estimating each user-story they now just count the user-stories and they know about how many user-stories they can do per sprint/iteration. This is done way easier and faster then estimating each story.

Estimates vs. forecasts

No estimates doesn’t mean you don’t have forecasts nor burn-down charts. You can do this based on the count of stories instead of story-point values. This further means that you don’t try to predict the future by guess-working but instead by measuring your work and doing a forecast based on that measured data (= number of user-stories).

Size of a story and slicing

Humans are very bad in estimating big tasks but we are better in doing so for smaller tasks.

In agile but also in no-estimates user-stories should be small very small. Not more then maybe two days. Even better not more then one day or maybe only a few hours. Ask yourself “Will I have this done by tomorrow?“. If the answer is “No” then you need to slice this story once more and make two out of it. If they still are too big – slice them again – and again.

Slicing stories and making each story smaller and smaller does not only lead to a precise “estimate” but also further clarifies the story. This means we reduce the over-all risk because when we slice it we also clarify the smaller peaces until we can easily handle them in our brain.

Another benefit of slicing is that you’ll get a 50% / 50% option to not implement a slice at all. This is very useful for managing the scope.

Managing the scope

Regarding to Vasco, managing the scope is the only way to fulfil deadlines successfully. Good project-leader/product-owners manage the scope from day 1. Having more but smaller user stories means that you as a PL / PO immediately have more options to manage scope for example by dropping one or another story.

From Ron Jeffrey’s I got the hint to do the stories with the most business value first (if possible). Together with a good DevOps pipeline and fast iterations you’ll push releases to customers very fast and hopefully get feedback fast too. As you go through your user-stories based on busines-values this ensures the right overall direction early in the project. You then can let the customer decide which release he/she likes to take to production. It’s also up to the customer if he wants to spend money for more iterations/sprints to get even more user-stories done. I know – this doesn’t work for each project setup. For example Gov projects don’t work like this even if they say they are agile they are not, never ever.

Offering fixed price projects

Let’s say you’re in the project business and have to offer fix priced projects on a regular basis. Projects of the size of about $200-500k need multiple days of work for estimation. Not counting the work to document them. It also needs the best employees to do it as it is really tricky and require a lot experience. Last but not least you will win only one out of a few projects. This means that this work mostly is done for free. A total waste of time in many cases but you must do offers to get work at some point.

How to do this more efficiently and with no estimates?

If you have to do a offer just compare the new project to your previous projects using this project catalog. This way you can tell the customer a rough price-tag very quickly. Propose to clarify further details in one (or more) workshops – which, by the way, are then paid.

Conclusion

Polyfill vendors and app bundle using Webpack 4

Thu, 09 Aug 2018 00:00:00 GMT

My goal was to have a polyfill.js bundle generated by Webpack 4 that gets inserted into the apps index.html file before any other Javascript as it must load and install the Poly- or Ponyfills before even vendor code runs.

This means I basically like to have the following budles loaded:

polyfills.js
vendors.js
app.js

I struggled with this and it took me quite some time to figure it out so I give it write.

Solution

First of all I added a file which does two things: a) reference the polyfill libraries I need to b) initialise them as needed. In my case I needed es6-symbol (for InversifyJS), es-promise and whatwg-fetch.

src/polyfill.js:

/* Load polyfill:
* This file and its dependencies get bundled into the "polyfills" chunk by WebPack and loaded as first scripts.
* Add loading code of further polyfills in here as needed. */

const es6_symbol = require('es6-symbol/implement');
console.debug(es6_symbol);

const es6_promise = require('es6-promise/auto');
console.debug(es6_promise);

import 'whatwg-fetch';

I then used this file as a second entry point besides my apps main file for Webpack 4:

entry: {
       polyfills: './src/polyfills.js',
       app: './src/index.tsx'
    },

Then I had to configure the output to use the junkname parts for the generated Javascript bundle files:

output: {
       filename: '[name].bundle.js',
       chunkFilename: '[name].bundle.js',
       path: path.resolve(__dirname, 'dist')
    },

With Webpacks 4 ‘s new optimization block I split all libraries from node_modules into a chunk/bundle called vendors:

optimization: {
        splitChunks: {
           cacheGroups: {
              commons: {test: /[\\/]node_modules[\\/]/, name: "vendors", chunks: "all"}
          }
       }
    }

This way I get polyfills.bundle.js, vendors.bundle.js and app.bundle.js. Then I use the HtmlWebpackPlugin to patch the scripts HTML tags into the index.html.

The problem with this is that the scripts-tags are generated in the wrong order so polyfills.bundle.js will not be loaded as the first package which results in errors. To address this I had to do quite a while of research so I figured one can change the chunk sort mode to manual and specify the chunks in the chunks field as a string array.

chunksSortMode: "manual",
chunks: ['polyfills', 'vendors', 'app'],

I could not find any official documentation about this option but finally got hints in the GitHub issues that this option exists and one hint how to use them.

Here is my the more complete webpack.config.js:

webpack.config.js:

const path = require("path");
const webpack = require("webpack");
const ExtractTextPlugin = require("extract-text-webpack-plugin");
const HtmlWebpackPlugin = require("html-webpack-plugin");
const CopyWebpackPlugin = require('copy-webpack-plugin');

module.exports = {
    entry: {
       polyfills: './src/polyfills.js',
       app: './src/index.tsx'
    },
    output: {
       filename: '[name].bundle.js',
       chunkFilename: '[name].bundle.js',
       path: path.resolve(__dirname, 'dist')
    },
    devtool: "source-map",
    resolve: {
       extensions: [".ts", ".tsx", ".js", ".jsx", ".json"]
    },
    module: {
       rules: [
          ...
       ]
    },
    plugins: [
       ...,
       new HtmlWebpackPlugin({
          template: "src/index.html",
          filename: "index.html",
          chunksSortMode: "manual",
          chunks: ['polyfills', 'vendors', 'app'],
       })
    ],
    optimization: {
        splitChunks: {
           cacheGroups: {
              commons: {test: /[\\/]node_modules[\\/]/, name: "vendors", chunks: "all"}
          }
       }
    }
};

State handling in React apps

Sat, 04 Aug 2018 00:00:00 GMT

Intro

We use React for most of our current web-frontends. For older React Apps we used the MobX (databinding-library for MVVM) and then moved over to Redux. But even with Redux we where not quite happy because we used Redux for nearly every button enable/disable state. We are also committed to TypeScript and its strong typings even for our actions and action-creators.

To get an impression what kind of code we wrote see this TypeScript Tutorial and this blog-post.

The problems

With this one ends up with a huge global state-tree maintained by a lot of reducers. Even small apps quickly had 20+ reducers. Each reducer serves multiple actions which needed their action definitions and their action-creators as you see in the TypeScript tutorial.

The main problem was not the amount of code but the maintainability and the readability of all this code. When one changed the user-interface one had to change the state-tree and all the actions and action-creators involved. If the UI-workflow changed more fundamentally, also the whole action and state-tree had to be re-thought, decoupled and refactored or re-built. This even in smaller apps (but with non-trivial UI) became a big issue sooner then later.

Another issue: for UI interaction we had to think in the event-sourcing pattern instead of a classic view-model (MVVM). Event-Sourcing is a good pattern but not really natural for UI-stuff like “when is which button enabled” or “what parts of the component are shown/hidden”. For this stuff its way over the top. Again, think of if EVERYTHING goes though action-creators, actions, reducers and finally into the state which then gets mapped to the React component properties.

We realised (thanks, chief) that we where way to slow implementing user-interfaces comparing to other stacks we used in the past.

Our solution

Four of us took on this topic and sat together, evaluated different things (even MobX again) and we then came up with the following:

Use Redux for global state – stuff that is used in different parts of the application.
Eg. central master-data fetched from the server, login-information, etc.
Use React component state for local state only.
Eg. if a component must maintain a enabled-disabled button-state this can be done easily in the local React component-state (the setState()-thing and must not go though all the Redux code.
Use the new React Context for related component state.
Eg. if a set of components are used to build a edit-form introduce a context on the top level component of the form and consume this within the sub-components like the soft-keyboard, advanced input-controls like editors, etc.

Its up to the developer to decide when to use Redux and when React Contexts. Its not a hard line. Think of Redux-State as the central singleton-store where the React Context is scoped state that gets dropped afterwards. Further more the context can be hierarchically cascaded. So if your state’s nature is more hierarchically and temporary you probably want to use the React Context instead of Redux. Same applies if you like to build a view-model with data and UI-logic in one place you’ll go with React Context. On the other hand, if things need to be persisted or you like to use history (eg. the Redux dev-tools) you will choose Redux.

The result

We practiced this at a few placed and quickly felt way more productive and used way less lines of code for the same UI. The result was especially mind-blowing when we changed a set of components that build up a complex value editor (used for touch-only devices used in a PWA) to use the React Contexts instead of Redux. We could remove a ton of code and the code get way more readable. Implementing new features to this value-editor became a relatively easy task. Using Redux the new editor-features require us to rebuild large parts of the state-handling code (and its tests) because actions, reducers and state is decoupled.

With the above three recommendations we quickly felt to be more productive in building and maintaining the UI while not loosing testability of our UI logic.

A few words about Redux

Don’t get me wrong. This is not a post against the use of Redux. We still like Redux and its Dev-Tools a lot. A really nice and transparent thing that makes debugging relatively easy. We don’t drop Redux entirely. We just don’t use it for everything anymore and therefore reducing its usage to a reasonable usage.

So, use Redux wisely, young Padawan 🙂

Structuring an app with React, Redux and TypeScript

Tue, 17 Jul 2018 00:00:00 GMT

I’m relatively new to the stack of React, Redux and TypeScript but I had to do a lot of coding in this area over the past months. To get started I read some books (yea old school – baby), got some nice and helpful intro by my co-workers and read a ton of online articles and tutorials.

But somehow it didn’t feel structured enough for me. So I revisited my code and where I’ve put things multiple times and slightly modified it on each attempt. With the current state I am quite happy bit its far from being perfect. So here is how I structure things as of today. No guarantee that I will be the same structure next week but hey, this is how the weg works 😉

Readings

First of all a few good reading everyone should read first.

A good reading for React/Redux apps is the Redux documentation. Several pages have very good information on how to structure things – even if they are not focused on TypeScript and type-safety. Each time a read – or better: re-read – chapters of this documentation I have a aha moment and think by myself “oh god, I’ll never get this all together in my code (a positive thing for the Redux docs).

Another good reading related to TypeScript is the TypeScript React Starter tutorial.

The result

Let’s start straight with the result. Here is how my files and folder structure currently looks like:

+ src
  + components
    + MyAppInputBox
      - MyAppInputBox.tsx
      - MyAppInputBox.less
    + MyAppPageLayout
      - MyAppPageLayout.tsx
      - MyAppPageLayout.less
    + ...
  + core
    + entities
      - IContract.ts
      - ...
    + api
      - ApiManager.ts
      - MyAppServer.ts
      - Fetcher.ts
      - IResponseObject.ts
      - ...
    + ioc
      - ServiceRegistry.ts
      - ServiceIdentifiers.ts
      - ...
  + redux
    + globals
      + actions
        - factories.ts
        - index.ts
        - types.ts
      + logic
        - serverInfo.ts
        - ...
      + reducers
        - globalsReducer.ts
        - ...
        - index.ts
      index.ts
      state.ts
    + reducers
      - index.ts
      - rootReducer.ts
      - pagesReducer.ts
    index.ts
    state.ts
  + pages
    + StartPage
      + components
      + redux
      + scences
      - StartPage.tsx
      - StartPage.less
    + ...

The index.ts files re-export the relevant members per folder. One can think of them as a kind of “published” visibility or the “public API of that folder”. They also allow for shorter import statements.

Components

There are many ways one can structure its React components and tons tutorials and articles can be found online. I like to divide my components in different categories depending on their role.

Simple components that are not connected to the global state and just do some rendering. They are located within pages or other “components* and are stored within a `componentsfolder. Other sources I found online call themelements“ but IDE’s – like WebStorm – have smart defaults for folder called components. That’s why I stick with this name.
Components that compose several simple components to a more complex layout and may are connected to the global state. The are pages or located in scenes subfolders. You’ll find other names for these components online like Views (a totally overused word in IT in my point of view), container-components, etc. I’ve started putting them into local scenes folders instead of local components folders.
Root level components that act as Pages or sometimes called Scenes. I find it handy to treat the entry point components for each new page (top-level component that renders the entire screen) separately. Located in src/pages or src/scenes.
Central components like input controls, logo’s, page-template etc. that are re-used all over in the application. I put these components in src/components.

This whole chapter is not yet in its final stage. Chances are quite good that I will further change the structuring of the React components in the future. But as I wrote – there are so many systems/concepts online for structuring React components.

Redux

Let’s talk about all the Redux related code and where I put those things.

First of all I normally have a src/redux where I put in my global Redux-related code which is not specific to a Page or Component. Things like my root state and reducers as well as setting up the Redux store itself goes in here.

More local sub-state – eg. the one specific to a certain page – goes into their own ./redux folder within the scope they belong to (Page, Component, etc.).

The Redux folders are always structured the same way:

+ actions
(+ logic)
+ reducers
- index.ts
- state.ts

State

The state.ts contains two things:

The interfaces describing the states and sub-states.
A getXyState(appState: IAppState): IXyState per state interface to get the given sub-state object out of the store. Using these functions the components etc. don’t need to know the structure of my reducers but only that fields are in the given state. So the only places where the cascading of my state-nodes matter are the combineReducers() and these getter functions.

A note on typing the states: I always use interfaces to describe / type the surface of a state. Others may use type aliases instead interfaces. I prefer interfaces.

Actions

Note: Actions are per state – not per reducer. The actions therefore are for all reducers within the given state node.

When using TypeScript the actions get more code then with vanilla Javascript because we like to type them and not using string literals all over. Then we need the factory functions (aka action creators) which produce the action objects. If one puts this all into one file per state this file quickly gets large. That’s why I split this code and introduced the actions folder.

./redux/actions/:

factories.ts: Contains all the functions that create the action objects (aka action creators). For example:

export function fetchContractsError(errorMessage: string): FetchContractsError {
return {
    type: FETCH_STARTABLE_CONTRACTS_ERROR,
    errorMessage
};
}

types.ts: Contains all the action type definition like shown on the TypeScript quick-starter tutorial and this is the only place where we use the string literals for action names:

export const FETCH_CONTRACTS_ERROR = 'FETCH_CONTRACTS_ERROR';
export type FETCH_CONTRACTS_ERROR = typeof FETCH_CONTRACTS_ERROR;
export type FetchContractsError = {
  type: FETCH_CONTRACTS_ERROR;
  errorMessage: string;
};

ìndex.ts: Pulls the above things together by re-exporting them.
```
export * from './types';
export * from './factories';
```

I’ve written Live-Templates for WebStorm for them so most of this code generated automatically. I just have to enter the new action name and fill it with additional fields.

I am thinking to change the content of the actions folder in the future the way that I have one file per action which contains the action-object typings and the factory-function (action creator). This way if I later need to move a action into another state-node I can grab the entire file and let WebStorm fix all the imports. But this would lead in even more files. As I said: thinking about it.

Reducers

The Reducers are stored within the reducers folders and are structured as shown in Redux docs and the TypeScript tutorial. Nothing special here except that we started putting the reducers into classes so we can use constructor-injection from the IoC container (we use Inversify and are happy with it). To get these reducers into Redux we write a wrapper-function per reducer which instantiates these classes using the IoC container (using the service-locator (anti-)pattern at this point) and can be used with the combineReducer() function of Redux.

Why do we need dependency injection (DI) in Reducers? We heavily rely on unit-testing (using Jest and WallabyJs). To do so we split our code using the SOLID principals into small classes that separate concerns. For example we extract domain-/business-logic into their own service-classes which we then register with the IoC container. To get instances of them we rely on constructor-injection and thats why we need classes and instantiate them using the service-locator pattern.

Logic folders

Not sure if this name will stay for long but it mainly contains stuff that don’t belong to the actions and not to reducers. For example async-actions (aka meta-actions) are stored here as they are not plain actions and also contain a decent amount of logic (eg. how to fetch server-data incl. error-handling). Logic utilities or sometimes the domain-/business-services goes in here as well. Its content is not highly standardized yet.

Core

The src/core folder contains client-side code that does not belong to either a specific React component nor a specific Redux state. Stuff like the fetch-client or domain-entities definition (= TypeScript interfaces) go in here. The whole IoC configuration is stored in here too as well as domain services and some other central code.

Summary

With this structure I feel kind of in control. But on the other hand we have a lot of files and folders and I always find things I don’t like. For example see the note on (re-)structuring the actions folder. This is why I declare this structure as work in progress. I wrote it down so you guys can give feedback and I can start over improving it – again 🙂

JavaScript async/await in React components

Tue, 20 Mar 2018 00:00:00 GMT

Again some notes from my recent React and Javascript/TypeScript learnings. I knew already how to use Promises by building method chains with .then() or .catch() to chain the asynchronous actions together. But as I am a fan of C#’s async and await and read that JS/TS does support them too, I liked to figure how to use them in my React apps so the code will get even cleaner as with the method chains.

Scenario

When my React UI need data they have to request them from a REST-Server (API). Best practice here is to do this work asynchronous so the UI-Thread stays responsive.

When speaking of React UI I mean a React.Component using its standard methods that like componentDidMount(). This method is where one should load the server-data with regards to the React doc’s. This method is normally not marked async and therefore one can not use await. I guess this is why many React developers don’t use await but method chains.

Solution

How can I utilise await to simplify my asynchronous code in React apps?

First I implemented an async service-method which will be responsible to fetch data from the server (for example by using the fetch() API which is based on Promises).

The following code is my in-memory test-service which returns fixed values instead calling a real server. Hint: I used the Promise.resolve() function to return the fixed value right away.

async getEntities(): Promise {
    return Promise.resolve(entities);
}

Note the async keyword here and that I return a Promise. Side-note for C# developers: Javascript Promise is exactly that Taskis in C#.

Then I’ve changed my React Component’s componentDidMount() mount method to an async method by also using the async keyword:

export class ContactList extends React.Component {
    ...
    async componentDidMount() {
        const entities = await apiManager().contactService.getEntities();
        this.setState({ entities });
    }
    ...
}

Please not that this way I can use await to await the result of my async function getEntities().

React seems to call this componentDidMount() even if it is async now.

Cool – eyy?

Compile-check Less CSS classnames using TypeScript and Webpack

Thu, 08 Mar 2018 00:00:00 GMT

As I am making myself familiar with modern frontend development based on React, TypeScript, Webpack and some more I learned something really cool. I like to write this down not only for you – dear reader – but also for my own reference.

The problem

Let’s say you have a trivial React component like this where you specify a classsName to tell what CSS class should be used:

const MyComponent = (props: MyComponentProps) => (

....

);

export default MyComponent;

The problem with this is that we don’t have any compiler-check to ensure this class myclass really exists in our LESS file. So if we have a Typo or later change the LESS file we can not be sure all classes/selectors are still valid. Not even the browser will show that. It silently breaks. Bad thing!

A solution

Using Webpack and the LESS loader one can fix this by check this at compile time. To do so you can define the style and its classname in the LESS file and import it into .tsx files. The LESS loader for webpack will expose the following LESS variables to the build process where the TypeScript loader (used for the .tsx files) can pick it up.

MyComponent.less:

@my-class: ~':local(.myClass)';

@{my-class}{
width: 100%;
background-color: green;
}
...

Note the local() function supported by the LESS loader (see webpack config at the end) which scopes that class to a local scope.

The above LESS files can be typed and imported into the .TSX file like this:

MyComponent.tsx:

type TStyles = {
myClass: string;
};

const styles: TStyles = require('./MyComponent.less');

const MyComponent = (props: MyComponentProps) => (

....

);

export default MyComponent;

Then firing your build the .less file gets picked up using the require() function and checked against the TypeScript type TStyles. The property myClass will contain the LESS/CSS classname as defined in the .less file.

I then can use the styles.myClass instead of the string literal of the original code.

To get this working ensure you have to LESS loader included in your webpack configuration (you probably have if your are already using LESS:

webpack.json:

module: {
rules: [
{
test: /.tsx?$/,
loader: "ts-loader"
},
{
test: /.less$/,
use: ExtractTextPlugin.extract({
use: [
{
loader: "css-loader",
options: {
localIdentName: '[local]--[hash:5]',
sourceMap: true
}
}, {
loader: "less-loader",
options: {
sourceMap: true
}
}
],
fallback: "style-loader",
...
}),
...
},
...

Note: The samples use LESS stylesheets but one can to the same with SCSS/SASS – I guess. You just have to use another loader for webpack and therefore the syntax supported by that loader.

No broken CSS classnames anymore – isen’t this cool? Let me know your feedback.

Calculating MD5 hash over multiple strings with C#

Tue, 06 Feb 2018 00:00:00 GMT

I just spent a fair amount of hours calculating one MD5 hash code over multiple strings so I thought I write this down. Please note that the same principle can be used for SHA1 and other algorithms provided by the .Net Framework. Please note that MD5 is not the strongest crypto. If you need strong security choose another algorithm.

To build one hash code for multiple data blocks one needs the TransformBlock() and the TransformFinalBlock() functions of .Net’s crypro API’s like the MD5 class.

Here is my main routine which calculates one MD5 hash over a whole set of ITranslationEntry. Each ITranslationEntry contains some string and a list of sub-classes (Texts) which strings too. I need one MD5 hash for the whole object hierarchy so I can detect changes in this structure which then needs to trigger some other logic.

using (var md5 = MD5.Create())
            {
                void AddStringToHash(ICryptoTransform cryptoTransform, string textToHash)
                {
                    var inputBuffer = Encoding.UTF8.GetBytes(textToHash);
                    cryptoTransform.TransformBlock(inputBuffer, 0, inputBuffer.Length, inputBuffer, 0);
                }

                foreach (var trans in defaultEntries)
                {
                    AddStringToHash(md5, trans.Category);
                    AddStringToHash(md5, trans.Key);
                    foreach (var transText in trans.Texts)
                    {
                        AddStringToHash(md5, transText.TwoLetterLanguageCode);
                        AddStringToHash(md5, transText.Text);
                    }
                }

                md5.TransformFinalBlock(new byte[0], 0, 0);

                return ConvertByteArrayToString(md5.Hash);
            }

The above code creates a instance of .Net’s MD5 class using its Create() factory method. Make sure you dispose this instance when not needed anymore as it uses operating-system resources. That’s why I put it in a using statement.

Next I have a local function called AddStringToHash() which does exactly what its name says by using the TransformBlock() method of the MD5 class. This method does two things. First id calculates the hash for the given block (the string in our case) and adds it to the MD5. Then it copies the input data to output buffer. I couldn’t figure out why it does this. But specify the input buffer as the output buffer does the job here.

Then we use this local function to add some of the property values when looping over our object hierarchy.

Finally we must call TransformFinalBlock(). Otherwise .Net will throw an exception when trying to access MD5.Hash.

The hash itself can be retrieved as a byte-array from MD5.Hash. As I need a proper string representation of it I added a little conversion method. Here’s the code:

private static string ConvertByteArrayToString(byte[] bytes)
        {
            var sb = new StringBuilder();
            foreach (var b in bytes)
            {
                sb.Append(b.ToString("X2"));
            }

            return sb.ToString();
        }

Hope this saves you some time if try to calculate one hash code for several data chunks.

How to build and publish NuGet packages with source symbols to VSTS

Thu, 11 Jan 2018 00:00:00 GMT

Note: This post originally contained 1 image that could not be recovered during the WordPress migration.

We read about that Visual Studio Team Services (VSTS) now not only support hosting your Nuget package feeds but also your source-symbols and source-symbol packages. This sounds very promissing in case you host your own Nuget’s und like to be able to debug them within your applications.

At my work we have an internal framework that is packaged in multiple Nuget packages. These packages are built by the VSTS Build-Job and get published to the VSTS hosted package feed in the Release-Job. It is consumed and used by our application using VSTS as a regular Nuget-Server.

As it was not trivial to get all running I do a quick summary here. Please also note, that a lot the Microsoft Docs are at least a little out of date. That seems to be the result of the new agile world we are living in.

Install package-management and enable source-symbol server

First of all you have to install the package-manager extension in VSTS. As source-symbol server feature currently is in preview you have to enable this feature-preview in the VSTS admin section. Read more about it here…

Index Sources & Publish Symbols

Then you compile your assembly’s by default you’ll get a .PDB file. This default is not only for debug- but also for release-builds. You can configure it the project settings of each assembly you have. Enable PDB generation. PDB’s are binary files containing debugging information.

To get enhanced mapping information pushed into the PDB’s you’ll need to add the build task “Index Source & Publish Symbols”.

Build-Tasks

Task Configuration

As you can see in the above screen-shoots I inserted this build task after building the solution and running the unit-tests. I set it to publish to the same VSTS account as the build-job runs.

For packing Nuget packages one can also configure to pack symbol-packages too. That means that not one but two packages will be generated by the Nuget-Task: the regular Nuget and the .symbols-Nuget package. So’ll see the important peaces here:

Build-Tasks

Nuget Task Configuration

That’s it for the build-job. Let’s go to deplyoment (aka Release-Job).

Deployment (aka Release-Job)

The good news here is that you don’t have to do anything to support source-symbol packages if you use the defaults of VSTS. The symbol-Packages get automatically picket up by the Nuget-Task if they exists in your build-artifacts.

Make sure you are using the newer Nuget-Task as the old one is deprecated anyway. Then make sure the .symbols.nupkg is in your path configuration. Again you can configure to publish it to your VSTS account’s package-management. I show you how it looks on my Release-job:

This should allow you to build and publish your own Nuget packages to your VSTS Package-Manager. You then can configure the package-manager as an additional source-server in your VisualStudio configuration. Read here how to do this.

Visual Studio for Mac with Ligatures font Fira Code

Thu, 11 Jan 2018 00:00:00 GMT

Note: This post originally contained 2 images that could not be recovered during the WordPress migration.

In a previous post I described how to set up the Ligatures font “Fira Code” in Visual Studio 2017. As I startet coding these days in Visual Studio for Mac (Version 7) I like to have Ligatures fonts in this IDE too. So I started a journey to set up “Fira Code” on my MacBook Pro with macOS HighSierra installed.

I had no clue how to install new fonts on a Mac. So I tried to download the TTF fonts and drop them in the folder /Library/Fonts without success. On the Fira Code Github site they write about click “Install Font“. This didn’t work too. Then I tried adding them via the app “Font Book” (part of macOS). Didn’t work.

Finally I used the method by installing it using the two “brew” command as described here as I have brew already installed on my Mac anyway. This finally worked!

The rest then was easy as Visual Studio for Mac supports Ligatures fonts out of the box:

Open Visual Studio for Mac
Open Preferences > Fonts
Choose Fira Core Retina as the editor font.
Done.

How to run xUnit tests in Visual Studio for Mac

Mon, 08 Jan 2018 00:00:00 GMT

Note: This post originally contained 1 image that could not be recovered during the WordPress migration.

In the current version of Visual Studio for Mac there is “Add file” template for xUnit Unit-Tests. Nice and easy. But how to run these tests? The problem is, that the default installation of Visual Studio for Mac does not discover these xUnit Tests even if you do a full rebuild.

After searching a while I figured that one need to install the extension called “xUnit.NET 2 testing framework support” from the Extension Gallery.

After that one need to restart Visual Studio for Mac and do a full rebuild. The tests will then show up in the “Unit Tests” pad.

Line breaks in webpages

Thu, 03 Aug 2017 00:00:00 GMT

I just got a bug-report where a field label is too long and (with the use of Bootstrap 3) doesn’t wrap but get cut off. So I took this opportunity to do some research about state of line-breaking in webpages and web-apps as of 2017.

CSS word-wrap

Since ages there is the CSS option called “word-wrap“. One can give it a value of “break-word” so the browser will break the text if it doesn’t fit into its box. For my Bootstrap 3 UI I put the following CSS in place for testing:

.form-group label.control-label {
    word-wrap: break-word;
}

Looks like it even works in crappy Internet Explorer.

Downside: The browser don’t know anything about break-points or even word-breaking rules from the language used. So they break long words anywhere.

HTML 5 word break-points

With HTML 5 we got a new HTML tag to specify where long words would be broken correctly. The Tag is like this

<wbr>

So one can use it like this:

This<wbr>is<wbr>a<wbr>very<wbr>long<wbr>word

If this is in place most browser will try to break at these break-points first before breaking anywhere else.

Downside: Internet Explorer does not support his at all but at least it doesn’t print the tag but ignores it.

Downside: The words get wrapped at the given places but no dash (“‘-“) will be shown so it still looks kind of incorrect.

Soft-Hyphens

Already the old HTML 3.2 standard defined a character called “soft hyphen”:

&shy;

The problem with this was that not all browser supported it. But the good news seems to be that today finally all browser – even Internet Explorer – added support for the soft-hyphen.

Using it:

This&shy;is&shy;a&shy;very&shy;long&shy;word

As it looks to me this solution doesn’t have any downsides yet so I will go with this one.

Visual Studio 2017 support for ligatures fonts like Fira Code

Tue, 25 Jul 2017 00:00:00 GMT

Note: This post originally contained 3 images that could not be recovered during the WordPress migration.

Based on the post ‘Monospaced Programming Fonts with Ligatures‘ from Scott Hanselman I tried if that works in for the classic Visual Studio too. Short answer: yes, the Ligatures support works out of the box – no settings other then choosing the font is needed.

Here is some sample TypeScript code written in Visual Studio 2017:

How to install?

Install the font on your Windows box like described here. Then open Visual Studio and go to Tools > Options > Fonts and Color and choose ‘Fira Code” and press ‘Ok’. That’s it!

Enjoy the new coding-world with Ligatures!

Estimations: Story Points vs. Hours

Sun, 23 Jul 2017 00:00:00 GMT

Note: This post originally contained 1 image that could not be recovered during the WordPress migration.

Over the last months the topic of estimations for developing software with many people arise many times. The thing with the story-points for estimations often seems to be confusing so lets write about it.

Why Story-Points?

When a software developers makes estimates in hours, others (Managers, Customers, Users, etc.) thinks of it as a more or less exact value. But the problem is, that it is really hard to name exact values in software development because each software and each project is different. So most of a developers daily business is doing things or the variation of it for the first time. Its more like an engineer / research style work then repetitive assembly-line style work. Try to ask scientist how long he needs until finished (in hours)?

So instead of estimate in hours the software industry came up with a new measurement: the Story-Points.

What are Story-Points?

All and nothing! Its a measurement that does not directly result in hours and its values have different meaning for each team! This is a very important thing: one can’t take the story-points definition from one team to another. Each team has its own understanding of the values used.

But really, what is it?

Each backlog-item (aka User-Story aka To-Do) gets a story-point value which describe the approximate size of the work. Backlog-items that are about the same size should have the same story-point value. When the team makes an estimate for a new backlog-item they think of which other backlog-item was baout the same size and give it the same story-point values.

Story-Points describe the relative size of a backlog-item to other backlog-items.

How to use Story-Points?

The questions is what we can do with the Story-Point values if they are no hours and a thing that only the one team understand?

The team works in sprints (aka iterations) of 2 to 4 weeks (defined by the team). After a few sprints the team knows how many Story-Points they can do in a sprint by sum the story-point values of the backlog-items they did per sprint. This values is called “Velocity”. Knowing this value allows the team to plan a new sprint by pulling in backlog-items until the total Story-Points reached their velocity (or even better: a little less). If team-members are on holiday etc. the team pulls in less story-points to address the reduced velocity.

Where come the hours into play?

After a team pulled in the backlog-items for a given sprint they plan the details for each of these backlog-items.

In tools like Microsoft Team Foundation Server (TFS) or Microsoft Visual Studio Team Services (VSTS) the team can add “tasks” to each backlog-item. The tasks are a relative small amount of work not more then a few hours. One can easily do an estimate for it in hours. All tasks together get the total hours needed for a backlog-item. All backlog-items together show the total hours for the upcoming sprint.

Then the team can define work-hours per day and days-off for team-members. This gives the total hours the team has for a sprint. TFS/VSTS can visualize all these values to show if a sprint (and their members) are on-track or go totally off.

Why not directly create tasks and add hours for all work?

Two things:

Lot of work: For a medium software project it would take weeks a team spend to get all the detailed estimates in hours. No project likes to spend all this time upfront.
Change of plans: In software projects its very likely that the plan will change. That’s why the industry came to “agile” software development but this would be another blog-post. Spending days or weeks making detailed estimates for work that will be dropped before it will started does not make any sense and is a waste of time.

What values to use for Story-Points?

The bigger a backlog-item is the more difficult it is for us humans to estimate exactly. To reflect this many teams use values from the Fibonacci number (1, 2, 3, 5, 8, 13, 21, …) only and limit the maximum value to certain value. Backlog-Items that are bigger then this maximum value must be spitted into two or more separate items.

But really: give me number!

Ok, lets see a sample what Story-Point values my colleagues and I often use:

1 = A trivial change with absolutely no risk, should not take longer then 30 minutes
e.g. changing a static text in the UI
2 = A small change with absolutely no risk, not longer then 4h
3 = Simple change with low risk and takes maximum 1d
5 = Medium change max. 2-3d or small change but with risk
8 = Larger change of max. 5d or change with risk
13 = Large change of max. 2 Weeks or with a decant risk

Risk: with risk I mean the risk a change takes longer then expected because we don’t know exactly how to do it or we for the first time.

Sprint length: we normally use a sprint length of three weeks. So a value of 13 with a max. of two weeks fits into a sprint well. Larger values then 13 don’t exists for our team because with our scale they would not fit into a three week sprint.

Update: I got feedback from multiple people that even “13” means to them: “We are really unsure about this item and will need to further split it for a better estimation”. So normally the items with 13 will be split to multiple items before work starts on these bigger items.

Trigger TFS and VSTS buildjobs using wildcards

Mon, 10 Jul 2017 00:00:00 GMT

In TFS and VSTS one can create a whole list of triggers that particular build-jobs. In the drop-down one can select an existing branch and all done. But if you like to trigger your CI build-jobs for all feature branches one can use wildcards too.

Eg. “feature/*”

To do so you have to expand the drop-down and enter the filter text into the Textbox that looks like a search box. Press ENTER and TFS / VSTS will use exactly this filter. Its not quite intuitive but works.

See the following screenshots.

Layer Architecture: Separation of View, Business Logic and Physical Data Access

Fri, 07 Jul 2017 00:00:00 GMT

Note: This post originally contained 1 image that could not be recovered during the WordPress migration.

This is a topic my colleague and I are discussing a lot the last few weeks because we started a little internal framework/library which should help us building small to medium vertical line-of-business web-apps using ASP.Net MVC. Its an old and widely discussed topic in the industry since years: what are your applications layers and what boundary should one use? Where does the presentation layer end, where or how is the business-logic accessed? How separated is it from the physical layer (data-access)?

There are a lot of layering- and separation-models. Then the more general strategies like Domain-Drive-Design, SOLID, well known patterns like Repository-Pattern or Unit-Of-Work come into play too.

Most topics result in the conclusion that one should not over engineer the solution and just do what the one application needs and what the customer ordered. That’s good if one just looks at one single application. But we have to do many such applications and therefore go further when specifying the architecture for the next few years and many apps. This then are hard to go for KISS (=keep it simple, stupid) – but we try.

Requirements

Knowing your requirements is always a good start. So what are our requirements?

The apps should use the same library and concepts so our devs can understand and maintain them easily. Currently each app has its own concepts and new devs need to be trained for each app.
Fit well for small to medium (10-60 Database Tables) line-of-business apps. They have a lot CRUD but also a decent amount of workflows / business-/domain-logic.
We not only build API’s but also standard entities and standard MVC UI for these standard entities. For example we have translation entities and optional a whole translation UI as part of the framework.
We currently work with Entity Framework 6 but will move to Entity Framework Core 2 one day or even move on to document DB’s like MongoDB or Azure Cosmos DB’s in the not so far future. So the physical data-access layer should be decoupled and we like to re-use as much logic and UI’s as we can when replacing the data-access layer.
The framework/library will be used for multiple apps over the next years as projects come to our table. (we are an IT-Service company in the project business)
Web-Services (REST etc.) must be supported and building SPA apps e.g. using React is on the horizon.
We saw that we have to force developers to a clear design/layering otherwise it will mess up (especially if external coders are in the team like near-/off-shoring).
We like to think of the database like yet another data-storage. We don’t use DB-specific features like stored-procedures etc. – only when there is absolutely no other way.
Business- / Domain-Logic must be fully unit-testet and stay in the .Net application code.
Don’t write our own level-level-framework but re-use existing libraries.
Use ORM’s (object-relational mappers) where possible. Don’t write SQL all over and don’t write our own ORM – even this would be fun 😉 Start with using Entity Framework 6.x and add support for EF Core and others later on – as needed.
Don’t just separate everything and don’t make every little change a huge effort through 10 layers. We don’t need an enterprise-grad layering (yet 🙂 ). KISS is in our mind.
The application must NOT be easily switchable from one to another backend (ORM etc) but must be able to choose one at the project start.
The framework/library must support the all supported backends (EF6/SQL, MongoDB/CosmosDB, EF Core, etc).
Layers we have in mind:
- Client-side Presentation (like a Javascript/Typescript ViewModel to do MVVM in the browser).
- Server-side Presentation (like some ASP.Net MVC view-models if needed)
- Business-/Domain-Layer (business-logic).
- Physical data-access (ready and write data into the database or table storage; ORM’s etc).

Layer-Variations

There are a couple of layer-variations when using an ORM. The following are those that make more or less sense to us. I guess there are about a trillion more variations one could create 🙂

Application Layers and their models / API’s

**Server-side View-Models:
**First a note about the server-side view models because I drawn them “a little special”. My experience is that sometimes one does not need to build a 1:1 view-model but can use the entities from the layer below. Sometimes I build a complete custom view-model layer and sometimes I wrap the entities or collections in a view-model and add more data-properties for the view but don’t rebuild all entity-classes. The later is my preferred approach when we talk about server-side view-models. This story is about the same for all the variations and can change easily.

**Type A:
**Basically define the EF/ORM Entities and use them all over – from repositories (data-access) to the representation. Its simple and every standard .Net / EF developer can participate from its first day on the project. The major downsides we discovered in our first alpha-versions is, that a) all data-backends need to support the single set of entities (EF, Mongo-Client all run on the same entity classes) and b) that not so experienced developers tend to place the business-logic etc. “somewhere” because they can. Layers are not really enforced. Both are no-go for us.

Type B:
EF/ORM entities are used for data-access in the repository and given to the domain-layer. Therefore the domain-layer does the mapping from the EF entities to the higher-level business objects. The apps talking to our Domain-Services does get the business-objects (maybe specialised per use-case) and don’t know anything about our ORM entities. A clean separation between presentation and persistence happens. If something changes on the physical-layer the app will not need to know this. The domain-layer can use the ORM entities and things like IQueryable to address complex data-access scenarios and have benefits of the ORM-mindset. On the downside it is more effort comparing to Type 1 as we have an additional model to build, map, test and maintain. It also can result in data-access code within the domain-layer that does not work with all supported ORM/repos.

Type C:
Domain-layer and data-access is completely separated with an additional DTO-layer. For me this is a theoretical variation and its again a lot more effort to implement and maintain. We will not go this way – I think.

Type D:
This one is similar to Type B but the business-objects go the the data-access layer and the data-access will map to ORM models (if needed at all). If offers great flexibility for implementing the data-access layer but as the mapping is done per data-access its more effort then Type B. An advantage is, that the domain-layer does know nothing about the data-access and has no possibility to run code related to data-access (and also is complete unaware of any ORM feature like change-tracking etc). Therefore advanced data-access scenarios that harder to address with Type D then with Type B.

Type E:
This type as a clear separation of the business-objects and the service-facade. The presentation does not know anything about the internals for the domain-layer or the layers below. Its all nicely separated but also a lot of effort to build and maintain (and test). For me this is more a enterprise-style layering and over-architectures for our small to medium apps.

Our final decision is not yet felt but we tend to go for B or D. What do you think? Did we miss a promising layering we should take into consideration too? Thoughts?

Microsoft Build 2017 Day 3

Sat, 13 May 2017 00:00:00 GMT

Auch am letzten Tag der BUILD 2017 Konferenz gab es wieder eine menge interessanter und unterhaltsamer Sessions. Einige waren so unterhaltsam, dass man diese schon fast als Nerd-Comedy bezeichnen könnte.

Entity Framework 2.0

Es kommen einige hilfreiche Erweiterungen. Z.B. das Mapping von komplexen Typen inklusive verschachtelten Typen sieht super gelöst aus. Freue mich darauf. Zudem ist mir aufgefallen, dass es möglich sein müsste, das neue, schlanke EF Core – danke NetStandard – mit “normalen” .Net-Full-Framework zu nutzen. Es müsste möglich sein, ASP.Net 4.6 mit EF Core zusammen zu verwenden. Das muss ich allerdings erst mal noch ausprobieren. Das wäre ein sehr cooler Stack für Projekte die noch nicht auf auf .Net Core/ASP.Net Core können, aber trotzdem schon das schlanke EF Core verwenden wollen. Beispielsweise für einfache Testbarkeit mittels In-Memory Driver oder dem komplett neuen SQL-Generator welcher schlankes, sauberes SQL erzeugt; im Gegensatz zu dem von EF6.

Zukunft von C#

Mit C# 7.1 (bis ca. 7.3) beginnt Microsoft damit für kleinere Spracherweiterungen auch mal einen minor Update zu liefern. Grosse Sprachupdates kommen dann aber voraussichtlich in C# 8.

Es wird viele hilfreiche Vereinfachung rund um Tuples geben. Einige diesen Features wie z.B. die Objekt-Deconstruction sind aber offen designed sodass sie auch für andere Anwendungszecke eingesetzt werden können.

Es wird auch einen Release (7.2) geben in welchem low-level Features eingeführt werden, welche z.B. für Xamarin interessant sind. Z.B. Datentypen auf Memory gemappt, sodass nicht immer wieder neu Memory alloziert werden muss.

Diverse weitere coole Sprachfeatures sind angedacht und werden aktuell konkretisiert. Der Inhalt von C# 8 ist aktuell noch nicht fixiert.

SignalR für ASP.Net Core

Der neue SignalR konnte leider nicht zusammen mit ASP.Net 2.0 in den Preview geschickt werden, da die Geschichte komplizierter und grösser ausfällt aus ursprünglich angenommen. So ist der kommende SignalR auch ein kompletter Rewrite mit einigen Änderungen auch im Konzept.

So ist der neue SignalR viel modularer und erweiterbarer aufgebaut als der alte. U.a. mit offenen Transport- und Hub-Protokollen sowie Formattern. Es wird auch möglich sein, den SignalR für andere Kanäle als HTTP zu verwenden. Für mich sieht das neue Teil extrem universell aus. Es scheint der richtige Schritt zu sein, hier etwas mehr Zeit zu investieren. Konkret rechnet das Team mit einem Release ca. Q3 2017.

Modernes Web Front-End Development mit VS 2017

VS lernt in der neusten Preview wieder einiges im Web-Tooling dazu. Es wird auch an neuen Templates gearbeitet die nicht nur mit .Net funktionieren (z.B. Static Websites) oder an optimierten Templates welche es sehr einfach machen sollen, z.B. mit VS2017 eine Angular2+ASP.Net WebApi Core App zu entwickeln. Auch IntelliSense kann jetzt viel mehr mit plain Javascript-Files anfangen da es neu auch JDoc Syntax versteht und die Metainformationen dieser Dokus verwendet. Z.B. mit dem aufgeborten TypeScript Kompiler und ESLint kann so die Entwickler-Experience für plain JS stark verbessert werden.

Dann gibt es da ein neues “Client-side Libraries” Features. Dieses erlaubt dem Entwickler Files aus unterschiedlichen Quellen (NPM, Bower, CDNJS, Filesystem) zu installieren – kein Packetmanager! Das Teil hat aus unserer Sicht Vor- und Nachteile. Wir sind uns hier noch nicht einige ob sich das Teil bewähren wird oder eher die Hintertür für Bastellösungen wird. Die Zeit wird es zeigen.

Das war es soweit von der BUILD 2017 aus Seattle.

Future of C#

Modern Web-Development with VS 2017

What’s wrong with this pic?

Microsoft Build 2017 Day 2

Fri, 12 May 2017 00:00:00 GMT

Heute war ich an diversen Session. Dazwischen max. 30 Minuten Pause – also ein dichtes Programm. Dabei habe ich die folgenden Themen aufgegriffen.

Windows 10

Diverse Updates wurden während der Day 2 Keynote für den kommenden “Fall Creators Update” gezeigt. Einige interessante Sachen und neue Programme wie ReMix sollen dabei sein.

Mixed Reality

Ist ein grosses Thema hier an der Build: Mixed Reality. D.h. man kombiniert die echte Realität mit künstlicher Realität. Z.B. (aber nicht nur) mit der HoloLens. Neu gibt es aus einen Kontroller – eine Art GamePad passend zur HoloLens. Thema wird ziemlich gehype’t.

.Net, .Net Core, ASP.Net und ASP.Net Core

In den QA-Sessions (Roundtable und .Net Open Q/A Panel) habe ich einige Punkte zu .Net, .Net Core, ASP.Net und ASP.Net Core gehört. Hier ein paar die mir wichtig erscheinen:

.Net Full Framework wird für immer da bleiben inkl. WinForms, WPF, UWP, ASP.Net etc. da es u.a. Teil von Windows ist. Würde es entfernt, wäre Windows kaputt was nicht akzeptabel ist. Hier besteht also kein Drang vom .Net Full-Framework zu wechseln.
ASP.Net Core 2.0 läuft aktuell nicht mehr auf .Net Full-Framework. Das Team analysiert die Kundenbedürfnisse, ob das noch gebaut werden muss.
Es könnte sein, dass ein paar ausgewählte Features von ASP.Net Core / MVC Core auf den klassischen .Net Full-Framework 4 Stack zurück portiert werden. Ist aber aktuell noch offen.
Windows Desktop Frameworks werden aktuell keine neuen Würfe geplant die einmal mehr alles über den Haufen werden würde. UWP bleibt also und wächst weiter. Ebenso WPF und – man staune – WinForms! Einige der neuen Features von Windows sollen auch in WinForms unterstützt werden.
Mit NetStandard 2.0 soll NetStandard auf Augenhöhe vom .Net Full-Framework sein.
Wo es Sinn macht werden Bemühungen vom .Net- und vom Xamarin-Team zusammen gelegt. z.B. neuer Xamarin-Linker

Xamarin

Die XAML-Dialekte werden aktuell als XAML Standard 1.0 vereinheitlicht. Analog NetStandard.
Xamarin erlaubt jetzt Multi-Prozess Debugging auch über Geräte hinweg. So kann man mit dem neuen Player die App direkt auf dem iPhone ausführen und via VisualStudio oder VisualStudio for Mac die App live debuggen (Remote-Debugging). In der gleichen Session kann man auch z.B. den Backend-Code debuggen.
Xamarin Forms wurde massiv ausgebaut und erlaubt eine echte Native-Experience.
Xamarin Forms wird jetzt auch auf macOs und Windows portiert – also nicht nur Mobile-OS wie iOS und Android.
Live Player für Xamarin Form UI’s direkt auf dem SmartPhone (iOS oder Android) welcher live auf dem Telefon alle XAML-Änderungen aus Visual Studio oder Visual Studio for Mac darstellt – also nicht im Simulator sondern live auf dem echten Telefon.
Der neue Embeddinator4000 erlaubt es, C#/.Net Assemblys als native Code zu kompilieren. Diese Libraries können dann im Zielsystem wie echte Libraries verwendet werden. Z.B: Object-C .framework oder Java Packete, etc.

Service Fabric

In einer Session zur Service Fabric wurde gezeigt, was das Teil so grob alles beinhaltet und das viele andere Azure Services darauf aufsetzen. Die Service Fabric kann auch on-prem betrieben werden und unterstützt neu Docker Container. Zudem wird sie neu als offizielle Container-Orchistrierungs-Software deklariert / verkauft.

Die Demos mit pro Minute 10’000 neue Windows Containers (=light VM’s) erstellen und 1’000’000 Linux Containers als ein Cluster zu betreiben waren echt beeindruckend.

Service Fabric muss ich mir definitiv genauer ansehen.

Visual Studio 2017

Mit ein paar Extensions kann VS bereits sehr nahe an R# aufgerüstet werden. Evtl. ist es mal wieder eine Versuch den R# abzulösen – wobei man dann allerdings mehrere andere Plugin’s verwenden müsste und ggf. noch eigene Code-Analyzers schreiben müsste.

Microsoft Build 2017 Day 1

Thu, 11 May 2017 00:00:00 GMT

Der erste Tag der Microsoft BUILD 2017 Konferenz ist für mich um. Zeit ein paar Highlights festzuhalten, was mir besonders geblieben ist. Dabei handelt es sich nur um einen ganz kleinen Ausschnitt aus der riesigen BUILD-Konferenz mit hunderten von Speakers und Tracks.

PostgreSql und MySQL als PaaS

Neu können auf Azure auch PostgreSql und MySQL als managed Datenbanken gebucht werden. Dies beinhaltet alle von Azure gewohnten Funktionen wie Skalierung und volle Wartung mit minimalen Downtimes.

Azure Cloud Shell

Das Azure Kommandozeilen-Interface (CLI) ist neu auch direkt im Browser via Azure Portal verfügbar. Dabei handelt es sich um eine echte Bash-Shell inkl. Cloud-Storage welche via Browser gerendert wird! Krass! Damit nicht genug: es gibt auch eine neue Azure Smartphone App für Admin. In dieser Smart-Phone App kann man ebenfalls die Azure Bash aufrufen und verwenden.

Azure AI

Die AI Suite (= künstliche Intelligenz) wurde wieder stark ausgebaut. Es gibt neue Services und für diverse bestehende Services wurden Konfigurierbarkeit eingebaut bzw. erweitert. Basis für alle AI sind u.a. die Daten – viele Daten. Ohne viele Daten keine AI. Diese Daten werden in sog. Graphen gespeichert. Ein Graph ist eine Sammlung von Nodes/Knoten welche untereinander Beziehungen haben. Ein Graph stellt eine Art riesiges Datengeflecht dar welches z.B. als Grundlage für die AI / KI dienen kann.

Azure Cosmos DB

Eine der grossen Ankündigungen war die Verfügbarkeit der neuen Azure Cosmos Datenbank. Nach 15 Jahren Entwicklung und Azure-interem Einsatz ist diese jetzt offen für alle Apps.

Azure Cosmos DB wird als die 1. weltweit skalierende, multi-model Datenbank bezeichnet. Die Datenbank ist von Anfang an auf massive, weltweite Skalierung ausgelegt worden und kann unterschiedliche Datenbank-Modelle (Key-Value, Document und Graph) bedienen. Darüber liegt ein stetig wachsendes Angebot an API’s. Aktuell u.a. DocuementDB, MongoDB, Gremlin, etc. Die Datenbankbank verfügt über fünf unterschiedliche Konsistenzlevel. Zudem gibt es SLA’s nicht nur für die Verfügbarkeit sondern auf für Antwortzeiten unter 10ms (lesen) und unter 17ms (schreiben). Die Datenbanken können auch grenzenlos wachsen und teile der Datenbank können mit sog. Geo-Fences versehen werden, sodass diese Datenbestände nur in den entsprechenden Ländern gespeichert werden. Für die lokale, offline Entwicklung gibt es den Cosmos DB Emulator.

Die Cosmos DB hat uns alle drei stark beeindruckt. Wir werden von der Cosmos DB wohl nicht das letzte Mal gehört haben.

ASP.Net Core 2.0 Preview

Die Session von Scott Hanselman und Daniel Roth war echt der Hammer! Viel Information und sehr guter Unterhaltungswert.

Ab sofort ist der Preview von ASP.Net Core 2.0 verfügbar. Dieser sowie auch der passenden Visual Studio 2017 Preview können side-by-side zu den aktuellen, stabilen Versionen installiert werden. Diese Version 2.0 basiert auf dem NetStandard 2.0 welche gem. den Herren jetzt Feature-Even mit dem .Net Full Framework ist. Die Anzahl API’s wurden somit nochmals massiv ausgebaut.

ASP.Net Core 2.0 bringt viele Vereinfachungen für den Entwickler mit sich. Zudem werden Standard-Funktionen auch übersicht in Helpers aufgelagert. Neu sind die Razor-Pages für einfache Seiten. Man muss damit keinen Controller, View und Model mehr separat machen sondern kann dass alles in einer Razor-Seite abhandelt. Praktisch man den Bedarf nach einer “einfachen Seite” innerhalt der ASP.Net Core MVC App hat.

ASP.Net Core 2.0 ist jetzt neu auch vorinstalliert mit .Net Core. Das führt dazu, dass die Apps viel schneller starten und auch ein Nuget-Restore sehr schnell von statten geht. Ausserdem muss für die App nicht mehr die ganzen System-DLL’s ausliefern was das Installationspaket sehr viel kleiner macht.

Visual Studio for Mac 1.0

Die erste offizielle Version von Visual Studio for Mac wurde freigegeben. Die IDE ist sehr ausgereift und hat bereits diversen Code (Editor Control, Roslyn, etc.) von Visual Studio 2017 übernommen. Man kann mit Ihr sehr gut Mobile-Apps (iOS, Android, tvOS, watchOS) wie auch Desktop oder Web-Apps entwickeln. Auch die Integration mit Docker und Azure-Publish ist bereits Realität. Visual Studio for Mac macht bereits einen sehr ausgewachsenen Eindruck.

Das mal so ein paar Eindrücke und Infos vom ersten Tag. Wer zu einem der Themen gerne mehr wissen will, sollte die nächsten Tage mal googeln. Die ganzen Detailinfos werden nah-dis-nah online gestellt.

Auto-complete Pull Requests with TFS and VSTS

Wed, 19 Apr 2017 00:00:00 GMT

Note: This post originally contained 2 images that could not be recovered during the WordPress migration.

At our company we are working mainly with TFS and VSTS using GIT repositories. We also set up our project teams with a Lead-Developer. A Lead-Dev has multiple responsibilities. One is to keep the technical overview of all of a projects source-code. We also are heavily committed on Code-Reviews. To do so we use TFS/VSTS pull-requests.

Each feature or bugfix is developed in its own GIT feature branch. A developer can commit to his/her feature branch as often as he/she likes. When the feature is done the Developer places a Pull-Request using the TFS/VSTS web-UI. Here is a good article on how one can place and handle pull-requests with TFS (but the current versions UI is much improved compared to the version showed in the article). This article is about the “Auto-Complete” feature of TFS/VSTS that I – as a Lead-Dev – really like and are getting used to it.

Review comments

When I get a pull-request from a Developer it shows me several tabs with all kind of information. On the first tab there is a general comment area – a kind of threaded chat. It also shows the summary of the comments left to the source-code files and updates to the pull-request. Think of it as activity-log of the PR – like Facebook for the PR.

One can also leave comments with the “Files” tab directly within source-code. Each comment has a status.

They are tight to a workflow: a reviewer can leave comments and a developer can look through them and mark their status accordingly. Note: once a pull-request is created, it can be updated by simply do a GIT PUSH to it. Using this technique the code get cleaned and comments can get addressed (or further discussed). If everything is fine a reviewer (or Lead-Dev) can “Approve” the code. If CI-Builds are fine too and the PR is linked to workitems (see TFS “branch policies”) the PR then can be “Completed”. The TFS-/VSTS-Server then will do a GIT MERGE (and optionally “squash” and delete the feature-branch).

Auto-Complete Pull-Request

As a Lead-Dev of multiple projects and libraries I spent some time in reviewing Pull-Requests. Most of them are of good quality but I have suggestions for improvments. I then can leave my comments as described above and instead of “Approve” the PR I can set it to “Waiting for author”. This means I wait on the fixes or answers from the Developer before accepting the pull-request. If there are no major complains I can set the PR to “Auto Complete”.

This means that if all lights are green (Comments are addressed, build succeed, work-items are linked. no merge-conflicts happen) the pull-request will be completed automatically asap.

This is a nice time-saver for me as a reviewer / Lead-Dev as I don’t have to re-visit and manually “Complete” this pull-request. The status is also visible to the Developers so they knows if they addresses the issues the PR will get integrated immediately. It also shortens the turnaround time so the PR is completed/integrated faster because they don’t have to get the Lead-Dev / Reviewer to review everything again. The is handy for the whole team as the team has the new code faster and less chance for merge-conflicts exists.

Thoughts about TagHelpers

Sat, 01 Apr 2017 00:00:00 GMT

ASP.Net MVC Core introduced the new TagHelpers. The should take the C#-Magic out of the Razor views so people without C# know-how – mainly “the HTML/CSS-only Web-Designer” can work on the views too with the tools they know and which don’t understand C#. Also the syntax looks nicer as the developers brain should not switch between C# and HTML all the time. The classic HTML-Helpers still exist. That’s what the advertisements say and it sounds good to me as a C# developer too.

Here is a sample for those of you never saw the TagHelpers:

Old HTML-Helpers:

@Html.LabelFor(m => m.MyProperty, new { @class="col-sm-2 control-label" })

New Tag-Helpers:

This label Tag-Helpers looks clean and nice.

My experience and thoughts

Now I worked with ASP.Net MVC Core for a while on my personal projects. After some ramp-up with the new TagHelpers it got easier – that was to be expected. I am far away from an expert or even have a deep background of the TagHelpers but what I noticed: many of them are not easier to work with as they need more brain-power to use as when using the Html-Helpers. But why is this? Ads told us it should be exactly the opposite! This made me think for a while now …

I noticed that the problem is with the non-trivial TagHelpers. Most TagHelpers are implemented as a custom HTML-Attribute (or a set of attributes) and not on Tag-Level. But this means I have to write the correct HTML-Tag myself and then use the Tag-Helper attribute. Further more this means that I as a developer have to remember two or three but one thing:

Which Tag-Helper exists?
How could the TagHelper be implemented so:
Which outer HTML-Tags must I surround what Tag-Helper with?

That’s a lot more brain-work (and brain storage space) I need then with HTML-Helpers:

Which HTML-Helpers exists?

Examples

Let’s look at an example where I really needed some time to figure out how the TagHelper works and had to ask stack overflow – couldn’t figure it out on my own: Validation Messages.

With a HTML-Helper straight forward:

@Html.ValidationMessage(m => m.MyProperty)

One basically needs brain-power to remember there is something with “….ValidationM….”. The rest is IntelliSense.

Now with Tag-Helpers:

<span></span>

Beside the fact that this is slightly longer I have to remember / know a few more things:

There is a TagHelper with “…validation…”.
For this helper I have to use a SPAN tag.
The SPAN Tag needs to be normal closed – not self-closing (which is the default of Visual Studio’s auto-complete).

When we come to the validation summary you have to know that you have to use it inside a DIV tag – not a SPAN because it builds a UL and LI ‘s which can not live inside a SPAN but can inside a DIV.

<div class="text-danger"></div>

But cool is the simplicity to support your own CSS classes. With HTML-Helpers they where as Microsoft did generate them with their own classes etc.

There are Tag-Helpers I still couldn’t figure out: @Html.EditorFor() or @Html.DisplayFor() (…and all the Editor-Templates).

The HTML/CSS-only designer guy

Then there is this often told HTML/CSS-only designer guy which has some basic HTML-Editor and knows about nothing more then HTML und CSS (and design hopefully). This guy then works on your view and stiles them.

I just never ever saw a HTML/CSS designer which made up even partially useable / maintainable HTML Markdown. IF you get HTML from a web-designer it mostly looks like shit. I’m always impressed that the browser get their crap looking good.

I talked to two React.js guys as they have a super cool new feature: JSX/TSX files. They are markup mixed with Javascript / TypeScript. …so Razor for the browser-client.

I asked them: “Does the HTML/CSS-only designer guys get this syntax? And what about tooling?“. Both answered: “Forget about this designer-guys – they do not exists like this! And tooling is ok. Its us developers how have to code the views anyway so use a powerful language.“.

Closing words

I know, TagHelpers are just one more tool in the Devs toolbox. You can use it but you don’t have to. BUT: There is huge push from ASP.Net Team to TagHelpers for all kind of things. Some of them are cooler and some of them – like the ValidationMessage.

To be honest, Visual Studio Tooling (with R#) was/is quite ok for the Razor views with Html-Helpers and HTML mixed. I saw cool usages of TAG-Level Tag-Helpers; not attribute-level TagHelpers.

I’m not yet a complete fan of them. I am thinking forth an back what the ideal “thing” in this are should look like. What do you think?

How to show IDs in TFS Storys and Tasks

Mon, 27 Feb 2017 00:00:00 GMT

Note: This post originally contained 4 images that could not be recovered during the WordPress migration.

I remembered that I saw TFS boards where the ID’s where shown directly within the board and one didn’t need to open each work-item to get it’s ID. I need these numbers to mention them in my commit messages so the commits and the work-items get link automatically by TFS/VSTS. As it took me quite some time to find these settings again I decided to write them down.

First of all we are talking about these little numbers here:

In TFS 2015 Update 3 they are hidden on the cards by default. To switch them on click the little gray gear icon next to the view filters. Not the gear icon in the header!

Using this gear icon one can configure the current view – and only the current view. If you like to have the setting on other views (eg. backlog vs sprints) too, you need to do it again in the other view(s).

Now you get a cool relatively new dialog where you can “Show ID’s” and configure other options like style-/color-rules for your cards and add custom fields etc.

I have configured some color-rules on my board to show story’s and task marked with some of my tags with other colors. For example “blocked” work-items are rendered as red cards.

Now, go and pimp your boards!

Removing unused NuGet packages

Sat, 18 Feb 2017 00:00:00 GMT

Note: This post originally contained 3 images that could not be recovered during the WordPress migration.

After working with Visual Studio and R# for many years now I discovered this feature just yesterday. Maybe some of you find it useful too.

I removed code that was depending on an external Nuget Package. Because the code is gone my project didn’t need the Nuget Package anymore. How to fast and safely remove the references and the package?

Let’s check the currently installed packages first:

Instead of opening the Nuget Packagemanager and uninstalling things manually there is faster way: right click on the project in the Solution Explorer and go into R# Refactor menu. There you’ll find the Remove unused references menu item.

This command will show you a dialog where one not only can select the plain assembly references to remove but also the references that come from a installed Nuget Package. This dialog shows a different icon for package references. See the highlighted item.

Clicking next will remove the selected references. For references that where introduced by an installed package the package is uninstalled too. Means your package folder is cleaned and the package is also removed from the package.config / .csproj.

The following show that not only the reference but also the package was removed:

Its a nice timesaver!

How to debug dynamic JavaScripts in Chrome

Wed, 08 Feb 2017 00:00:00 GMT

Note: This post originally contained 2 images that could not be recovered during the WordPress migration.

If your app is loading HTML with Javascript in there using AJAX these dynamic scripts will not show up in Chrome script-debugger (nor Firefox). They show up in IE 11 and Edge but without name.

To be able to debug them you can name these scripts using a comment on the end of the script block:

//# sourceURL=PersonMassUpdateTab.js

Place this //# sourceURL=....js code just before the closing script tag.

After loading this content you’ll see the dynamic script in Chrome’s debugger like this:

Edge will show it too with name but IE doesn’t:

Where is the package.config file gone?

Sat, 04 Feb 2017 00:00:00 GMT

I just updated to the latest VisualStudio 2017 RC, .Net Core 1.1.1 and worked on a previously started little ASP.Net Core app. I’ve managed Nuget packages for the app by adding and removing packages. When it came to commit my changes I thought the version control must be broken: no changes made to the package.config file each .Net developer got so used to over the last years.

After a while I took a closer look at the issue. The package.config does not exist anymore. But the .csproj file exists again (no project.json anymore) and its format changed. It’s short, really short and no GUID’s in anymore. And hey: here we go – there are the packages, right within the .csproj! Here is a complete .csproj:

netstandard1.6
  
  
    library
  
  
    
    
    
  
  
    
      ..\..\..\..\..\..\Users\marc.duerst\.nuget\packages\microsoft.aspnetcore.identity.entityframeworkcore\1.1.0\lib\netstandard1.3\Microsoft.AspNetCore.Identity.EntityFrameworkCore.dll
    
    
      ..\..\..\..\..\..\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.dll

See the PackageReferences? So nice and clean. Not only 3rd party packages are in there but also the NetStandard library itself. I also remembered I read about this change a while ago. Microsoft re-invented the .csproj files and dropped the project.json files introduced with .Net Core 1. More infos here

R.I.P. package.config !

SignalR Connection Indicator Reloaded

Wed, 01 Feb 2017 00:00:00 GMT

Some time ago I’ve written a blog post on how to build a connection indicator using ASP.Net MVC, SignalR and TypeScript. I had to write one again and did an improved version of the previous one by better using Knockout.js.

The final result looks like the initial graphic. So the connection indicator will change its color and text to reflect the online state of the app.

It also uses a view-model that is bound using databinding to the DOM elements. So we are talking of a MVVM pattern here. As the basic principal still applies we can go straight to the code.

SignalR Hub

First of all I implemented a very simple SignalR Hub in C# using Visual Studio 2015 “Add new item” dialog:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using Microsoft.AspNet.SignalR;

namespace WebApplication1.Hubs
{
    public class ConnectionStateHub : Hub
    {
        public void Hello()
        {
            Clients.All.hello();
        }
    }
}

Razor Layout

This time I’ve integrated the connection indicator in the Razor layout _Layout.cshtml so its rendered on all my pages. The layout is a standard Bootstrap 3 page with Knockout.js loaded. The important part of code is this one:

<div class="navbar-header pull-left connectionStateNav">
    <span id="connectionStateContainer" style="font-size:18px" class="hidden">
        <span class="label label-default">
            <span></span>
        </span>
    </span></div>

Its basically a Bootstrap label with a span inside it. I use just these two elements with databinding for Knockout.js. The outer label binds its CSS class to connectionStateCssClass while the inner Span binds its text to text to connectionStateText.

At the bottom of the _Layout.cshtml I have my TypeScript (Javascript) linked:

http://~/signalr/hubs

My external script file gets initialized using the following Javascript code at the bottom of the _Layout.cshtml:

$(function () {
    window.connectionState = new MyApp.Core.OnlineStateViewModel();
    ko.applyBindings(window.connectionState, $(".connectionStateNav").get(0));
    window.connectionState.startSignalRHub($.connection);
    $("#connectionStateContainer").removeClass("hidden");
});

It loads my TypeScript code from MyAppOnlineState.js and the SignalR hub from the virtual reference signalr/hubs. Then it create a new instance of MyApp.Core.OnlineStateViewModel and apply the Knockout databinding using the data- attributes for all elements within my connection state DIV. Finally the SignalR Hub gets started and the connection indicator shown by removing the hidden class.

Viewmodel OnlineStateViewModel

Now you probably as for the MyAppOnlineState.js which is a TypeScript file so its source-code is in *MyAppOnlineState.ts. Here we go:

namespace MyApp.Core {

    export class OnlineStateViewModel {
        private connectionStateHub: any;
        private subscriptions: Array<Function> = [];

        public connectionState = ko.observable(4);
        public connectionStateText = ko.observable("");
        public connectionStateCssClass = ko.observable("info");

        public subscribeStateChange(callback: Function): void {
            this.subscriptions = this.subscriptions.concat(callback);
        }

        public startSignalRHub(connection: any) {

            this.connectionStateHub = connection.connectionStateHub;
            this.connectionStateHub.client.hello = () => {
                console.info("SignalrR: hello() received.");
            }

            connection.hub.stateChanged((state: any) => this.connectionStateChanged(state));
            connection.hub.disconnected(() => {
                setTimeout(() => { connection.hub.start(); }, 10000); // Restart connection after 5 seconds.
            });
            connection.hub.start();
        }

        public connectionStateChanged(state: any): void {
            const stateConversion = { 0: "Verbinde", 1: "Online", 2: "Neu verbinden", 4: "Offline" };
            const stateClass = { 0: "label-default", 1: "label-success", 2: "label-warning", 4: "label-danger" };

            console.log("SignalR state changed from: " + stateConversion[state.oldState] + " to: " + stateConversion[state.newState]);

            this.connectionState(state.newState);
            this.connectionStateText(stateConversion[state.newState]);
            this.connectionStateCssClass("label " + stateClass[state.newState]);

            this.subscriptions.forEach(f => f(state));
        }
    }
}

The viewmodel basically sets things up in startSignalRHub. Important here is the assignment of the eventhandler stateChanged calls the connectionStateChanged() function. This one calculates new values for the properties connectionState, connectionStateText and connectionStateCssClass.

The viewmodel also features a subscriber mechanism where other scripts can add them so they get notified if a connection state change happens. They can do so by calling the function subscribeStateChange() with the callback to call in case of the connection state changes.

Common Service Registry

Fri, 23 Dec 2016 00:00:00 GMT

Based on my previous blog post “Decouple IoC/DI container when authoring a C# library“ I have written some C# Nuget packages to have that Service Registry thing easily available in all my C# projects. The package(s) are called “Common Service Registry” (see here).

Packages

Currently the Common Service Registry (CSR) contains three packages. You mainly need two of them. The one always needed is the CommonServiceRegistry.Core. It contains the basic interfaces like ICommonServiceRegistry against which one needs to register its DI needs.

To finally use the Common Service Registry one also needs an adapter package that build the bridge between the concrete IoC product you are using and the Common Service Registry.

The following two adapters are currently available:

CommonServiceRegistry.SimpleInjector for using CSR with Simple Injector (my favorite)
CommonServiceRegistry.Unity for using CSR with Unity

So you need the Core package and one of the above adapter. If you need an additional adapter please create an issue over on GitBub.

How to use

Check out the how to use over at GitHub. Its really easy to use that thing. The application needs one additional line and the library needs no additional lines of code (comparing to register without Common Service Registry).

Writing an adapter

A new adapter can be implemented with minimal effort. See the source-code of the other adapters (eg. this one). Note that each adapter also should have its unit-test assembly and that there is a base-class for adapter tests. So writing unit-tests for new adapters is done with a few lines of code as you can see here.

Contribute!

Contributions are very welcome. If you like to check out (or fork) the source-code you’ll find it on here GitHub. You can have a look at the issues and pick one to work on.

Testability for Dynamics CRM SDK Code - Part 2

Wed, 14 Dec 2016 00:00:00 GMT

In Part 1 I showed how I code against the Dynamics CRM SDK with my own ICrmSession and how I can do unit-testing based on those few interfaces and classes. In this second part I’d like to cover some more topics the reader may find interesting.

Testing ExecutionRequests

When coding agains the Dynamics CRM SDK from outside of CRM (like for data import/update services) one likes to do multiple data updates within one atomic transaction. The only way I found to do is to build an ExecuteTransactionRequest and fill its Requests collection with CreateRequest and UpdateRequest instances. Then fire this ExecuteTransationRequest using OrganizationContext.Execute() method.

Here is some sample code I’ve built on top of my ICrmSession as discussed in my previous post. Its a simplified version of my code just to point out the basic principe how I send Insert’s and Update’s to Dynamics CRM. Code like this I have used in my base-class for Upsert-Services (one per Entity).

public virtual OrganizationResponse DoUpsert(ICrmSession session, TUpsertData upsertData, string entityIdentification, IQueryable<TEntity> query, Func<TEntity> initNewEntity, Action<TEntity, TUpsertData> updateEntityFields)
        {
            if (session == null) { throw new ArgumentNullException(nameof(session)); }
            if (upsertData == null) { throw new ArgumentNullException(nameof(upsertData)); }

            var request = new ExecuteTransactionRequest
            {
                Requests = new OrganizationRequestCollection(),
                ReturnResponses = true,
            };

            // Update existing
            var entityExists = false;
            foreach (var existingEntity in query)
            {
                entityExists = true;
                ...
                updateEntityFields(updateEntity, upsertData);
                updateEntity.Id = existingEntity.Id;
                updateEntity.EntityState = EntityState.Changed;
                request.Requests.Add(new UpdateRequest { Target = updateEntity });
            }

            // Insert connection-object if no one did exist
            if (!entityExists)
            {
                var newEntity = initNewEntity();
                updateEntityFields(newEntity, upsertData);
                request.Requests.Add(new CreateRequest { Target = newEntity });
            }

            try
            {
                return session.OrganizationService.Execute(request);
            }
            catch (FaultException<OrganizationServiceFault> ex)
            {
                // TODO: Log the exception
                throw;
            }
        }

As discussed the main point is to build a ExecuteTransactionRequest containing CreateRequest and UpdateRequest instances. Then I fire it using session.OrganizationService.Execute(). The important thing here is that I use my ICrmSession to get the organization service.

With this code in place I can easily unit-test without the need for a real Dynamics CRM connection. To do so I can re-use my CrmSessionTestBase class I showed already in the previous post:

public abstract class CrmSessionTestBase
{
    public CrmSessionFake CrmSession { get; private set; }
    public IOrganizationService OrgService { get; private set; }

    [SetUp]
    public void SetUpCrmSession()
    {
        CrmSession = new CrmSessionFake();
    }

    [TearDown]
    public void TearDownCrmSession()
    {
        CrmSession?.Dispose();
        CrmSession = null;
    }
}

This unit-test baseclass creates a fake of ICrmSession which records the sent organization requests using a dynamic fake built with the Fake It Easy library. Here is the CrmSessionFake code. For more information read the previous post.

using System;
using System.Collections.Generic;
using FakeItEasy;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Sdk.Client;

namespace CrmTests
{
    /// <summary>
    /// Fake of a <see cref="ICrmSession"/> that can be used for unit-testing.
    /// </summary>
    /// <seealso cref="ICrmSession" />
    public class CrmSessionFake : ICrmSession
    {
        public CrmSessionFake()
        {
            OrganizationService = SetUpOrganizationServiceFake();
            OrganizationServiceContext = SetUpOrganizationServiceContextFake();
        }

        /// <summary>
        /// Gets the <see cref="OrganizationRequest"/>'s that where sent using <see cref="OrganizationService"/>
        /// or <see cref="OrganizationServiceContext"/>.
        /// </summary>
        /// <value>The sent <see cref="OrganizationRequest"/>'s.</value>
        public IList<OrganizationRequest> SentOrganizationRequests { get; } = new List<OrganizationRequest>();

        /// <summary>
        /// Gets the organization service.
        /// </summary>
        /// <value>The organization service.</value>
        /// <seealso cref="IOrganizationService"/>
        public IOrganizationService OrganizationService { get; }

        /// <summary>
        /// Gets the typed organization service context.
        /// </summary>
        /// <value>The organization service context.</value>
        /// <seealso cref="ICrmSession.OrganizationServiceContext"/>
        public OrganizationServiceContext OrganizationServiceContext { get; }

        /// <summary>Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.</summary>
        public void Dispose()
        {
        }

        IOrganizationService SetUpOrganizationServiceFake()
        {
            var result = A.Fake<IOrganizationService>();

            // Record all call OrganizationRequest's sent to .Execute() in SentOrganizationRequests
            A.CallTo(() => result.Execute(A<OrganizationRequest>._))
                .ReturnsLazily((OrganizationRequest request) =>
                {
                    SentOrganizationRequests.Add(request);
                    return new OrganizationResponse
                    {
                        Results = new ParameterCollection(),
                        ResponseName = Guid.NewGuid().ToString(),
                    };
                });

            return result;
        }

        OrganizationServiceContext SetUpOrganizationServiceContextFake()
        {
            var result = A.Fake<OrganizationServiceContext>();
            A.CallTo(result)
                .Where(call => call.Method.Name == "CreateQuery")
                .WithNonVoidReturnType()
                .CallsBaseMethod();

            return result;
        }
    }
}

The important point here is that each unit-test that inherits from CrmSessionTestBase has a property CrmSession which is of type ICrmSession (interface). It is set up with a fake so it records all call to OrganizationService.Execute() within the property SendOrganizationRequests for later use.

So my final unit-tests are as simple as this:

[TestFixture]
    public class MyUpsertServiceTests : CrmSessionTestBase
    {
        [Test]
        public void Test_InsertNewEntity()
        {
            // Arrange
            var upsertData = CreateUpsertData();
            var service = new MyUpsertService();

            // Act
            var organizationResponse = service.Upsert(CrmSession, upsertData);

            // Assert
            ValidateEntity<CreateRequest>(organizationResponse, upsertData);
        }

        [Test]
        public void Test_UpdateExistingEntity()
        {
            // Arrange
            var upsertData = CreateUpsertData();

            // Call to OrganizationContext.CreateQuery() would return a fix (aka existing) is_contractaccount
            A.CallTo(CrmSession.OrganizationServiceContext)
                .Where(x => x.Method.Name == "CreateQuery")
                .WithReturnType<IQueryable<MyEntity>>()
                .ReturnsLazily(() => new List<MyEntity> { CreateEntity(upsertData) }.AsQueryable());

            var service = new MyUpsertService();

            // Act
            var organizationResponse = service.Upsert(CrmSession, upsertData);

            // Assert
            ValidateEntity<UpdateRequest>(organizationResponse, upsertData);
        }

Again, this is a simplified version of my code as I did a generic implementation of this for re-use with all of my Upsert-Services but it should show the important parts.

Two things to point out here:

The Test_UpdateExistingEntity() highjack the query execution by faking the result for calls to OrganizationServiceContext.CreateQuery(). This is done with Fake It Easy again. See here for more samples and a quickstart with Fake It Easy. I highly recommend getting used to a smart fake/mock library like Fake It Easy. It can help you so much for unit-testing.

The second thing is the ValidateEntity() method. I use it to check if the right requests would have been triggered by my upsert service. Means, I am checking if CreateRequest or UpdateRequest are triggered and with what entity data (upsertData). First my implementation which needs some Reflection hacking:

protected void ValidateEntity<TExpectedRequestType>(OrganizationResponse organizationResponse, TUpsertData upsertData)
    where TExpectedRequestType : OrganizationRequest
{
    organizationResponse.Should().NotBeNull();
    CrmSession.SentOrganizationRequests.Should().HaveCount(1);
    CrmSession.SentOrganizationRequests.First().Should().BeOfType<ExecuteTransactionRequest>();
        ((ExecuteTransactionRequest)CrmSession.SentOrganizationRequests.First()).Requests.Should().HaveCount(1);
    ((ExecuteTransactionRequest)CrmSession.SentOrganizationRequests.First()).Requests.First().Should().BeOfType<TExpectedRequestType>();

    var request = (TExpectedRequestType)((ExecuteTransactionRequest) CrmSession.SentOrganizationRequests.First()).Requests.First();

    // HACK: Get the entity out of the "Target" property
    var requestType = request.GetType();
    var propType = requestType.GetProperty("Target");
    var entity = (TEntity)propType.GetValue(request);

    ValidateEntityContent(upsertData, entity);
}

The first few lines validate and extract the requests from CrmSession.SentOrganizationRequests. No Rocket Science here.

Then the tricky part is to get the entity data out of either CreateRequest or UpdateRequest in a generic way. The problem here is, that both have a property called Target which holds the entity but sadly the Dynamics CRM SDK API is not very well crafted here. The base class of CreateRequest and UpdateRequest is the general purpose OrganizationRequest which does not have a Target property. There is no common base class like TargetedOrganizationRequests. The Microsoft Team just copy-pasted the Target property to both classes. Even worst, they made both classe sealed so they can not be “fixed” from outside outside by re-introduce at least a shared interface for the Target property.

Long story short: the only way I was able to address this using generic code (without using copy-paste) was to use the Hack with Reflection. Its a shame but better then copy-paste like Microsoft Dynamics did.

Updating and restoring data on Dynamics CRM during unit-tests

Sometimes the above faked CRM-connectivity are not enough. The problem is that you only test what you code will fire against Dynamics CRM but you don’t test the reaction of CRM. For example if Dynamics CRM for some reason don’t accept the data you send this would not be covered with the above kind of unit-tests.

To address these kind of scenarios I added some tests that do more of an integration-tests kind of unit-tests (black-box instead of white-box testing).

To do so I use a baseclass for my unit-tests that does not use CrmSessionFake but a real CrmSession instances. These tests then connect to the projects DEV-Instance of Dynamics CRM Server and do real data-update.

The problem here is again that the CRM SDK greatly laks of clean transaction support. There is no *begin-commit-rollback” transaction style API. So the common pattern to just open a database transaction before the testing code runs and then always rollback the transaction at the end of the unit-test doesn’t work here.

I address it in a more classic scrappy way by doing a try-finally and delete the data I’ve just created again. This requires a few things so it is at least a little bit stable:

Each test-run should create its own unique set of data so the clean-up code can identify the data it needs to delete.
Update of existing records can not be easily be undo (eg. if other testrun runs in parallel). So create your unique test-data before and then test the updating with the create test-data.
Build re-usable helpers to clean up the data.

My pattern for these kind of tests goes like this:

// Generate testdata
var updateData = new MyData{ MyNumber = $"Test_{Random.Next(9999)}" };
GenerateUpdateData(updateData);

try
{
    // Run testcode which updates the real CRM data in here ...
}
finally
{
    // Clean up
    RemoveEntities(s => CreateQuery(s, updateData.MyNumber ));
}

And finally my little helper method RemoveEntities which I placed in my unit-tests baseclass:

protected void RemoveEntities<TEntity>(Func<ICrmSession, IQueryable<TEntity>> queryBuilder) where TEntity : Entity
{
    using (var session = SessionFactory.CreateSession())
    {
        foreach (var entity in queryBuilder(session))
        {
            session.OrganizationServiceContext.DeleteObject(entity, true);
        }

        session.OrganizationServiceContext.SaveChanges(SaveChangesOptions.ContinueOnError);
    }
}

It gets a ICrmSession from my session factory and calls the queryBuilder delegate (Lambda etc.) to get a query. Then it loads the results from the query and deletes each of them using OrganizationServiceContext.DeleteObject(). Finally the changes get saved using the SaveChanges() method.

Conclusion

If one spends a few hours once to introduce a few basic classes and interfaces things later get easy to work with. In my case having the ICrmSession and “SessionTestBase class opens up a whole lot of possibillities. So when starting with a new project take your time – you will easily save it later on.

Testability for Dynamics CRM SDK Code

Wed, 26 Oct 2016 00:00:00 GMT

During the last months I had to code several web-cervices (ASP.Net WCF Service) for connecting other pieces of software with Dynamics CRM. I’ve used the Dynamics CRM SDK (2011, 2015 and 2016) to access CRM from within my C# code.

For those of you who don’t know the Dynamics CRM SDK: it’s a bunch of .Net libraries (and tools) you can use in your C# code to query and update data and metadata in Dynamics CRM.

It provides not only one way to access CRM but about three or four. One is a typed Linq API similar to the EntityFramework where one uses generated C# classes and a fluent API. Another one is accessing CRM untyped using string identifiers for entity and fields names. The third option is to build a transaction style API where one build a transaction request object contains create and update requests. One then executes this request using the SDK and CRM will (try) executing all the contained create and update statements within one transaction.

Tip: For code running outside of Dynamics CRM the transaction request allows you to get multiple write-operations executed within one transaction.

Problems

Connecting to CRM is not quite easy and if you are connected you get multiple objects for accessing CRM (typed and untyped API’s). There is no clean session object.

As unit-testing my code is a must have for my daily work I quickly ran into the issue how to test my Dynamics CRM code. In the first spot this seems tricky as your code highly depends on the existence of Dynamics CRM. There are some caveats in SDK itself like sealed classes, missing interfaces and missing common base classes.

Session-Handling

As a start for my upcoming business logic I like to have a nice disposable session object which provides access to miscellaneous parts of the CRM SDK. I also like to have a easy to use factory for creating sessions (connections). All of this should be ready for the inversion of control (IoC) pattern using Depencendy Injection (DI). More precisely for using constructor injection which is my prevered way of how to do DI.

This means that I have to use C# interfaces and constructor parameters requesting the interfaces they need and depend on.

Let’s code!

First, the Session interface and implementation which holds the SDK’s OrganizationContext and OrganizationService:

ICrmSession Interface:

/// <summary>
/// Wraps the xRM <see cref="IOrganizationService"/> and <see cref="OrganizationServiceContext"/>
/// in a session that gets disposed correctly.
/// </summary>
/// <remarks>Make sure you use <c>using</c> or call <see cref="IDisposable.Dispose"/> manually!</remarks>
public interface ICrmSession : IDisposable
{
    /// <summary>
    /// Gets the organization service.
    /// </summary>
    /// <value>The organization service.</value>
    /// <seealso cref="IOrganizationService"/>
    IOrganizationService OrganizationService { get; }

    /// <summary>
    /// Gets the typed organization service context.
    /// </summary>
    /// <value>The organization service context.</value>
    /// <seealso cref="OrganizationServiceContext"/>
    OrganizationServiceContext OrganizationServiceContext { get; }
}

Then the implementation of ICrmSession which also implements the IDisposable interface. Note that my code uses C# 6 features. If you target an older version of C# you have to do some minor changes to the code.

CrmSession Implementation:

/// <summary>
/// Wraps the xRM  and 
/// in a session that gets disposed correctly.
/// </summary>
/// 
/// 
/// 
public class CrmSession : ICrmSession
{
    public CrmSession(IOrganizationService organizationService, OrganizationServiceContext organizationServiceContext)
    {
        if (organizationService == null) throw new ArgumentNullException(nameof(organizationService));
        if (organizationServiceContext == null) throw new ArgumentNullException(nameof(organizationServiceContext));

        OrganizationService = organizationService;
        OrganizationServiceContext = organizationServiceContext;
    }

    /// 
    public IOrganizationService OrganizationService { get; private set; }

    /// 
    public OrganizationServiceContext OrganizationServiceContext { get; private set; }

    /// 
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected bool IsDisposed { get; private set; }

    protected virtual void Dispose(bool disposing)
    {
        if (!IsDisposed)
        {
            OrganizationServiceContext?.Dispose();
            OrganizationServiceContext = null;

            (OrganizationService as IDisposable)?.Dispose();
            OrganizationService = null;

            IsDisposed = true;
        }
    }

    ~CrmSession()
    {
        Dispose(false);
    }
}

Nothing too fancy so far. Now we need a factory to easily create such ICrmSession instances. First we need an interface again.

ICrmSessionFactory Interface:

public interface ICrmSessionFactory
{
    /// <summary>
    /// Creates a new session and connects to the CRM instance.
    /// </summary>
    ICrmSession CreateSession();
}

And now the implementation of ICrmSessionFactory. It creates the OrganizationServiceProxy and OrganizationServiceContext CRM SDK which are used to connect to CRM. it also configures the support for the typed proxy classes and finally construct a CrmSession instance with the all of this.

CrmSessionFactory Implementation:

/// <summary>
/// Implements <see cref="ICrmSession"/> against Dynamics CRM.
/// </summary>
public class CrmSessionFactory : ICrmSessionFactory
{
    readonly Uri defaultOrgUri;
    readonly ClientCredentials defaultOrgCredentials;

    public CrmSessionFactory(ICrmSessionFactoryConfig config)
    {
        if (config == null) { throw new ArgumentNullException(nameof(config)); }

        defaultOrgUri = config.ServerUri;
        defaultOrgCredentials = config.Credentials;
    }

    internal CrmSessionFactory(Uri defaultOrgUri, ClientCredentials defaultOrgCredentials)
    {
        if (defaultOrgUri == null) { throw new ArgumentNullException(nameof(defaultOrgUri)); }

        this.defaultOrgUri = defaultOrgUri;
        this.defaultOrgCredentials = defaultOrgCredentials;
    }

    /// <inheritDoc/>
    public ICrmSession CreateSession()
    {
        var organizationServiceProxy = new OrganizationServiceProxy(defaultOrgUri, null, defaultOrgCredentials, null);
        var organizationServiceContext = new OrganizationServiceContext(organizationServiceProxy);
        organizationServiceProxy.EnableProxyTypes();

        return new CrmSession(organizationServiceProxy, organizationServiceContext);
    }
}

The code above should look familiar to CRM developers. As our factory does have a constructor that takes a configuration object we now need the interface and implementation for that. Later on we can use this configuration object to configure our factory instances right from the DI container.

ICrmSessionFactoryConfig Interface for easy configuration of the factory

public interface ICrmSessionFactoryConfig
{
    Uri ServerUri { get; }
    ClientCredentials Credentials { get; }
}

CrmSessionFactoryConfig Implementation

public class CrmSessionFactoryConfig : ICrmSessionFactoryConfig
{
    public CrmSessionFactoryConfig(string orgUrl, string orgUsername, string orgPassword)
    {
        if (string.IsNullOrWhiteSpace(orgUrl)) { throw new ArgumentNullException(nameof(orgUrl)); }
        if (orgUsername == null) { throw new ArgumentNullException(nameof(orgUsername)); }

        ServerUri = new Uri(orgUrl);
        Credentials = new ClientCredentials
        {
            UserName =
            {
                UserName = orgUsername,
                Password = orgPassword,
            }
        };
    }

    public Uri ServerUri { get; }
    public ClientCredentials Credentials { get; }
}

So far the above code is generic. We can but it into a central library for reuse in other apps and projects. Build a Nuget package for it allows you to easily update it later on.

Example on how to use the sessions

Now I can create and use the session like this:

var config = new CrmSessionFactoryConfig('<orgurl>', '<orgusername>', 'orguserpwd');
var factory = new CrmSessionFactory(config);

using (var session = fatory.CreateSession())
{
    var query session.OrganizationServiceContext
        .CreateQuery<MyEntity>()
        .Where(p => p.Firstname == "John")
    ...
}

Note: After leaving the using scope the CrmSession and therefore the connection to the CRM gets disposed automatically.

This code is clean but did not save a lot of lines of code. But we now have a clear separation in place (decoupling).

Example for using with Dependency Injection (DI)

As we have everything decoupled nicely we now can inject them using Dependency Injection. To do so add a DI container and configure it somewhere in your applications startup. The following code show how to register a `ICrmSessionFactoryConfigsingleton instance and theICrmSessionFactory“ using SimpleInjector.

var container = new Container();
container.Register<ICrmSessionFactoryConfig>(() => new CrmSessionFactoryConfig(crmwsConfig.OrgUrl, crmwsConfig.OrgUser, crmwsConfig.OrgPassword));
container.Register<ICrmSessionFactory, CrmSessionFactory>();
// TODO: Add other type registrations here
// TODO: Use this container to create the instances of your root objects (see container documentation)
...

Note 1: Using other DI containers like Unity, DryLock, Ninject, etc. is basically the same but may vary in the syntax.

Note 2: Beside creating and configuring the DI container you must make sure that the instances of your objects do get created by your DI container so the constructor injection is in place. How this is done depends on the type of application and which DI container you are using. See your DI container documentation and samples for More information. There probably is a Nuget package for your container and type of application to set up these things automatically.

With this registration in place I’ve build my service classes containing the logic like this:

public interface IMyService 
{
    void Serve(string name);
}

public class MyService : IMyService
{
    private readonly ICrmSessionFactory sessionFactory;

    public MyLogic(ICrmSessionFactory sessionFactory)
    {
        if (sessionFactory == null) { throw new ArgumentNullException(nameof(sessionFactory)); }

        this.sessionFactory = sessionFactory;
    }

    /// <inheritdoc />
    public virtual void Serve>(string name)
    {
        using (var session = sessionFactory.CreateSession())
        {
            var request = new ExecuteTransactionRequest
            {
                Requests = new OrganizationRequestCollection(),
                ReturnResponses = true,
            };

            var newEntity = new MyEntity() { 
                Name = name,
            };

            request.Requests.Add(new CreateRequest
            { 
                Target = new MyEntity() 
                { 
                    ... 
                } 
            });

            session.OrganizationService.Execute(request);
        }
    }

Note: I also registered IMyService with my DI container so one level up
(for example in your WebAPI controller) you can request an instance of MyService using constructor parameters too.

public class HomeController()
{
    private IMyService myService;

    public HomeController(IMyService myService)
    {
        if (myService == null) { throw new ArgumentNullException(nameof(myService)) }

        this.myService = myService;
    }

    public ActionResult Index()
    {
        myService.Serve(...)
        ...
    }
}

Unit-Testing

The above was pretty straight forward and gives us a nice layer of abstraction and decoupling. With this in place Unit-Testing of your service logic in `MyController“ can be done by faking or mocking the ICrmSession. This way we do not need a Dynamics CRM running and get lighting fast unit-tests.

To do so I created myself a mock-class for ICrmSession which I can reuse in other apps and projects. It basically catches all call to OrganizationContext.Execute() and records them in the property SentOrganizationRequests. To set up the fakes for IOrganizationService
and OrganzationContext I use my favorite faking library Fake it easy.

CrmSessionFake Implementation

using System;
using System.Collections.Generic;
using FakeItEasy;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Sdk.Client;

namespace CrmTests
{
    /// <summary>
    /// Fake of a <see cref="ICrmSession"/> that can be used for unit-testing.
    /// </summary>
    /// <seealso cref="ICrmSession" />
    public class CrmSessionFake : ICrmSession
    {
        public CrmSessionFake()
        {
            OrganizationService = SetUpOrganizationServiceFake();
            OrganizationServiceContext = SetUpOrganizationServiceContextFake();
        }

        /// <summary>
        /// Gets the <see cref="OrganizationRequest"/>'s that where sent using <see cref="OrganizationService"/>
        /// or <see cref="OrganizationServiceContext"/>.
        /// </summary>
        /// <value>The sent <see cref="OrganizationRequest"/>'s.</value>
        public IList<OrganizationRequest> SentOrganizationRequests { get; } = new List<OrganizationRequest>();

        /// <summary>
        /// Gets the organization service.
        /// </summary>
        /// <value>The organization service.</value>
        /// <seealso cref="IOrganizationService"/>
        public IOrganizationService OrganizationService { get; }

        /// <summary>
        /// Gets the typed organization service context.
        /// </summary>
        /// <value>The organization service context.</value>
        /// <seealso cref="ICrmSession.OrganizationServiceContext"/>
        public OrganizationServiceContext OrganizationServiceContext { get; }

        /// <summary>Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.</summary>
        public void Dispose()
        {
        }

        IOrganizationService SetUpOrganizationServiceFake()
        {
            var result = A.Fake<IOrganizationService>();

            // Record all call OrganizationRequest's sent to .Execute() in SentOrganizationRequests
            A.CallTo(() => result.Execute(A<OrganizationRequest>._))
                .ReturnsLazily((OrganizationRequest request) =>
                {
                    SentOrganizationRequests.Add(request);
                    return new OrganizationResponse
                    {
                        Results = new ParameterCollection(),
                        ResponseName = Guid.NewGuid().ToString(),
                    };
                });

            return result;
        }

        OrganizationServiceContext SetUpOrganizationServiceContextFake()
        {
            var result = A.Fake<OrganizationServiceContext>();
            A.CallTo(result)
                .Where(call => call.Method.Name == "CreateQuery")
                .WithNonVoidReturnType()
                .CallsBaseMethod();

            return result;
        }
    }
}

Then I created a little reusable base class for all my tests which sets up a new CrmSessionFake for every test and disposes it on tear down of each test:

CrmSessionTestBase:

public abstract class CrmSessionTestBase
    {
        public CrmSessionFake CrmSession { get; private set; }
        public IOrganizationService OrgService { get; private set; }

        [SetUp]
        public void SetUpCrmSession()
        {
            CrmSession = new CrmSessionFake();
        }

        [TearDown]
        public void TearDownCrmSession()
        {
            CrmSession?.Dispose();
            CrmSession = null;
        }
    }

Finally code the test for MyService:

using NUnit.Framework;

[TestFixture]
public class MyServiceTests : CrmSessionTestBase
{
    [Test]
    public void Test_InsertNewEntity()
    {
        // Arrange
        var service = new MyService(CrmSession);

        // Act
        service.Serve("John")

        // Assert that a CreateRequest was set up and fired corretly
        CrmSession.SentOrganizationRequests.Should().HaveCount(1);
        CrmSession.SentOrganizationRequests.First().Should().BeOfType<ExecuteTransactionRequest>();
        ((ExecuteTransactionRequest)CrmSession.SentOrganizationRequests.First()).Requests.Should().HaveCount(1);
        ((ExecuteTransactionRequest)CrmSession.SentOrganizationRequests.First()).Requests.First().Should().BeOfType<CreateRequest>();

        var request = (CreateRequest)((ExecuteTransactionRequest) CrmSession.SentOrganizationRequests.First()).Requests.First();
        var entity = request.Target;

        entity.Name.Should().Be("John")
    }
}

Remark: In practice the logic in MyService is slightly more complex then in this example so testing it makes a lot more sense and there may be more then one simple test.

Conclusion:

My main point besides providing some code samples is that even with the CRM SDK one can (and should) do unit-testing for your C# code. The most important one to test in every application is the business- / domain-logic. Its where the business value comes from and makes your application worth living.

You can go even further then the above samples. Some ideas (and work in progress at my company):

Implement helper methods in your test baseclass to do standard test validations like the SentOrganzationRequests.
Introduce more abstraction from the CRM SDK by wrapping the API’s you need in their own interface.
When writing CRM plugins and workflow-activities extract your logic out of your plugin code. Decouple them from the SDK plugin method.
In plugins and workflow activities wrap your OrgContext and Service in a ICrmSession too and write your code against this interface

Just some ideas.

Go and do it! Its cool coding work – way better then do manual testing all the time. Spend you time in automate things once instead in doing manual work (testing) over and over again.

You are a developer – a coder – not a click-user! 🙂

C# Regions are Evil

Mon, 03 Oct 2016 00:00:00 GMT

The programming language C# supports the use of regions since ever. To be more precise it dates back into VS .Net 2003 and .Net 1.x. The concept is fairly simple: use the keyword #region and #endregion to mark a block of code as a collapsible and optionally named region.

History

Where do the regions come from? They started with C# 1 when the language did not have that much language features. Not only Generics where missing but also partial classes didn’t exists.

In these early days tools like Visual Studio did use the regions to wrap their generated code. This way the IDE by default collapsed the code generated from tools like the Forms-Designer.

With C# 2.0 Microsoft introduced the feature called partial classes. With this feature a class can span more then one file so the tools could generate their code into an entire separate file. No need for regions anymore.

Additionally Microsoft added foldable code sections based on the semantics of the code into the text editor. This way a developer can collapse methods, IF-Statements, classes, namespaces, etc.

Today

I am surprised how many developers still use Regions in their C# code even in 2016. Some years ago I tend to use regions too. That time seniors asked me: “Why are you using regions?”. My immediate answer was to clean the code and increate its readability. Shouldn’t one structure its code the way collapsable code (aka regions) is not needed at all? From then on I challenged myself for each use of a region: “This should go into its own method (or even its own class). Should it?”. The result was, that I never ever wrote a region again.

Problems with regions

So what are the problems with regions? Why the hell do I write an entire blog article about them? Even in 2016 developers around the world (or at least around Europe) are still using them. This made me think about the problems with regions:

Regions are painful for other developers inspecting your the first time. They first have to expand all the collapsed sections to get the full picture of the code. And because of all the #regionand #endregion lines the code gets even larger.
Regions often lead to bad code because developers easily put a region around larger code blocks to make them hide instead of thinking how to structure their code well.

The second issue is really bad. Stuff that leads in writing bad code hurts me. Regions are one of these things.

I find usages of regions mainly in these places:

Grouping members in a class (private, public, etc.

If a class contains that many members so collapsable grouping is needed the class probably needs to be split into multiple classes. With large classes chances are hight that they are responsible for too many things. See the “Single Responsibility Principle” from SOLID.

On the other hand: if a class only serves a single responsibility it is probably small enough so it doesn’t need regions for grouping its members.

Using regions here lead in bad design of the classes as they handle way too many things. For small classes regions are just a waste of code-lines. No need for regions here.

Regions within methods

That’s another area I regularly find regions. When methods get a little longer some developers tend to group the sub-parts of a method into regions. I did so too. The idea is that one can collapse them and just read the region-names to get the big picture of the method.

As you may guess: bad design too. Not only clean-code suggest to write short methods so maintenance and testability of them is good.

Instead of using a region here one can easily extract a new method with the given block of code. Sometimes extracting its own class is even a better match. If a developer cares about the “Single Responsibility Principle” the classes don’t get that complicated so methods tend to be less complex and shorter too.

So again: using regions here leads in bad code.

Collapse generated code

As written above: since C# 2.0 there are partial classes for this. Use them instead regions!

What to do with regions

It would be great if Microsoft can remove them from the language features but this isn’t really an option as it will break a lot of existing code. So the best thing is to not use them anymore.

So far so good but what with existing code?

One of the first Visual Studio extensions I install on my dev machines is the “I hate regions” extension. It auto-expand all regions and renders them in a small and grey font so the don’t disturb that much.

Another thing one can do is to set up a R# rule and apply it to the entire solution. Bye-bye regions.

So, dear developers – don’t use regions anymore. Let them go! R.I.P Regions!

Quick developer journey on Microsoft Application Insights

Fri, 29 Jul 2016 00:00:00 GMT

Note: This post originally contained 4 images that could not be recovered during the WordPress migration.

I am currently working on a new little web application using the good old ASP.Net MVC 5. This application should be hosted on Azure App Service and as our company is highly committed to the Azure Cloud we also have all kind of subscriptions in place. When Visual Studio 2015.3 asked me whether to active the “Application Insights” I quickly decided to do so and created the Azure ResourceGroup as well as set up AppInsight as part of the “New Project” wizard of Visual Studio. I thought this way I can get familiar with AppInsights.

After coding some days on the app I recognised a red square in my code today followed by the text “2 Exceptions”:

“aw I must click this thing!” came immediately to my mind. Did so:

Wait, what? “2 (new)” … ah got it: “2 new NotImplementedException at TfsProjectWriter.CreateProject. The method or operation is not implemented.“. Ok, not to surprising as I am currently in the middle of implementing this application and I ran it quite some times already on my dev machine. But App Insight seems to record these things already and there are these cute little links again one just can not resist to click.

As a developer I first clicked “Go to code”. It took me directly to the code of the “CreateProject” method which shows me the problem. Same thing when clicking the blue method name itself:

Ok, no surprise here and code is always good. But what where those other cute little links?

View all exceptions in this app

This one takes you to a Visual Studio panel where you can query all reported exceptions users experienced in your application. One can sort, filter, drill-down and more – all kind of querying of exception data directly within Visual Studio. You can drill-down and get all the information about the exceptions including inner-exceptions and interactive stack-trace etc.

There are tons of filter options. Not only the exception info but also which user from what location, hardware etc. The kind of information from browser HTTP requests.

From the exception details one can query other exceptions wich may relate based on criteria filtering of the source exception. For example check other exceptions of this user in that timeframe or check if other users had the same issue (called “Problem”). This way you can get the context why the errors happened so you can fix it easily.
Nice but what is the Trend thing. Sound promising too.

Trends

The trends links open another Visual Studio panel which shows kind of a multidimensional cube with all possible data about request, responses, errors, performance, etc. One can again filter, group, sort, visualise them in many possible ways – just as you would except from a cube. All straight within Visual Studio just two clicks away from your actual code! The following screenshot show some of the options.

You can query all possible things – again using request and application data. You have a ton of options. To make the start with that Cube a little easier there are pre-build queries one can click and get the cube pre-configured:

Azure Portal Integration

The Application Insights are nicely integrated into the Azure Portal. So one can query things from outside of Visual Studio within any web-browser:

Application Insights Portal

If you are a real hardcore analytic you may like to get even more query options. So you can point your web-browser to the Application Insight Portal and get the full blown thing. Here you can even write your own queries:

And did you hardcore analytics see the ultimate hard-core option: “Export to PowerBI“. Have fun!

Downside of Application Insight

Yes, there is a downside. More exactly there are two downsides:

There is no on-prem version – it’s cloud-only.
At the time writing this Application Insight Data can only be hosted in the U.S. No option to host it at least on continental europa.

This means you need to closely consider privacy for each of you apps, project and customer. For me these two options are a no-go for at least half of the projects/customers.

Integration options for your App

You can integration the collecting of Application Insight data in more then one way.

Every web-application can include a Application Insight Javascript and collect data via browsers. As this is a simple option and open for all kind of web-apps it can not collect server-inside data.

For this you can use the Application Insight SDK on the server and log your stuff. If you are a ASP.Net developer Microsoft has the option already in the stack and you just have to activate it. If you like to get more then the default data you can add SDK-Calls to your server-code too. Same for the client-code using their Javascript-SDK.

So far I didn’t explore the options for native-, desktop- and mobile-apps but I am pretty sure one can use the SDK and fire the log-requests from those platforms too.

Conclusion

Besides the obvious pros and cons it’s the first time I can watch my code to be alive when users are actively using it. Especially with the “Live” views it’s like you see your users using your software. It’s kind of feeling you software. Cool stuff. But there is the privacy side too. Sure it’s anonymized but somehow – especially the live-viewing – feels a little bit wrong to my Swiss culture. Maybe I just have to get used to it.

But all in all its cool tech and I like to explore it more when real usage data will come in.

Git Pull Requests and Reviews in TFS/VSTS

Fri, 22 Jul 2016 00:00:00 GMT

Note: This post originally contained 11 images that could not be recovered during the WordPress migration.

GIT Pull-Requests and Reviews in TFS/VSTS

TFS or its cloud instance Visual Studio Team Services (VSTS) does have a very nice integration of GIT as a distributed version control system (DVCS). This is build into TFS since TFS 2015 and did extend with every release of TFS. The TFS Version Control backend (TFVC) is still available but my first choice when I create a new source-code repository is GIT.

How to create a GIT repo in TFS

This is a snap. Just go to your TFS / VSTS Team Project open its settings page using the gear icon in the upper left corner.

Open the Tab “Version Control”.
Press “New pository…”
Choose the “Type”. Default is already GIT.
Give it a name and hit “Create”.

Note: Each Team Project can have only one TFVC repository but an unlimited number of GIT repositories. One of the advantages when using GIT repos.

Set up your branches for pull-requests?

Go again to the V_ersion Control_ settings of your Team Project and choose the branch you like to configure and switch to its Branch Policies:

Here you can configure the branch as you like. Check out the zoom-in of the options – quite impressive I would say but wait until you see them in action:

First you can force pull-requests to the given branch and specify a build job that gets triggered as the developer sends in a pull-request. You can configure how often the build job runs and if the pull-request gets blocked if the build job is not yet run successfully. I prefer to block it so I force everybody in the project to have builds running which includes unit-tests must passing.

Then you can configure if the pull-request must be linked with work items and if it should be blocked or not. I normally don’t block them as this gets hard for checking in fixes etc. But I active it so I will get an indicator showing me if the pull-request is linked or not. As a lead-dev I then can consider contacting the team member to ask for work items.

A really cool thing are the review options. Configuration is pretty self explaining. You’ll see it in action later on.

Finally you can configure who of your team members can or must review what part (path) of your repository. So if someone for example checks in code in a core library path maybe the lead-dev for the library should review that change too.

Now what?

With the above settings in place your team-members can not push changes directly to the master branch anymore. If they try they will get an error. Instead they have to create feature branches, push them and when done with the feature send a pull-request for this feature branch. Think of it as a “request to pull the changes from the feature branch to the target branch” (normally into the master branch). The lead-dev (or who ever is in charge) can check this pull-request and decide if he likes to integrate it the master branch.

Create a pull-request

For the demo I’ve created a little change in feature branch feature/blog_test and pushed it to the GIT server (aka GIT remote). As you see TFS will just prompt me to create a pull-request for it.

Note: If you edit a file directly in the TFS web UI and try to push it to the master branch TFS will prompt you to right away create a feature branch for you and push it include creating a pull-request. Cool!

I can eighter continue work on the feature and push more commits to it or I can create a pull-request now.

Note: You can create the feature branch our own using the GIT client of your choice or you can use the TFS web-ui to do it. For example you can create a feature branch directly from within your product backlog item / user story. This will link the feature-branch automatically with your work-item. The feature branch can also be created from the Code page. Many options to do it.

So let’s create a pull-request now by clicking the New pull request button in the TFS web-ui.

Here you can check and complete the information about this pull-request. You may wanna update the description and link it with work item(s). You also can specify the reviewers you like to get reviews from (or leave the defaults in place).

Click New pull request and the pull-request will be sent to the team and reviewers. Your job as Dev for this feature is done now. Except: you like to get the review’s done so your feature gets the DoD (Definition of Done) pass.

How to integrate the pull-requests?

So, what do have so far after the developer created the new pull-request you may ask? We still have the developers feature branch and we have a pull-request which requests to integrate this feature branch into the master branch (or how else you called your branch). TFS also fired the build job specified in the branch policies on the feature branch. The reviewer got notified that there is a pull-request ready for review. So far nothing changed on the master branch – that’s important.

Now, when you (the Lead Dev, Reviewer or Team-Member with appropriate permissions) go to the Code > Pull Requests you’ll see the pending pull-requests.

Note: On the top you have filters like Mine, Active, Completed and Abandoned. It took me a while to find them.

When you click on a pending pull-request you’ll get the details for this request:

I like that UI but its not obvious how it work when you see it the first time. So I highlighted some interesting stuff.

In the middle column you see the pull-request header (from/to branch etc). Just below you again get filters / tab-pages. By default the Discussion tab is active but you can switch to File and Commit to check those details too.
On the top right you’ll see a list of indicators depending on your branch policies. Things like if merge-conflicts would happen, review-status, work-item assignements and – if configured – the build-status. Below there are the two possible actions: Complete pull-request and Abandon. We’ll get to this in second.
Further more on the right side you have the Review area. You see who reviewed with what feedback and you can give your own feedback. You also can leave comments in the Discuss area in the middle and even cooler you can leave your review-comment directly in the source-code view on per line base:

Now, if everything is OK with the pull-request you can Complete pull request using the button on the top left.

You’ll then get asked for some details:

Here you can finally fix up subject and description and have to nice GIT shortcuts:

Delete after merge will delete the feature branch after it was successfully merged into the target branch (here master). Folks pulling/syncing will get it deleted too but in case there tracking the branch locally they have to delete their local branch manually. But at least they see that it is gone on the remote server already. Nice timesaver.
Squash changes uses the GIT “Squash” feature to compact the history. All the commits in the feature branch will be packed into one single commit in the master branch. So we “squash” all commit together to be one single commit in the history of the master branch. It’s a matter of taste how you like to have your history look like. Again: nice timesaver.

Now the final act is to press the Complete merge button. The TFS server will fire all the GIT commands to integrate it into the master branch and – if checked – delete the feature branch. If you have continuous integration build-jobs and even continuous deployment release-jobs active the will be fired right now on the master branch – just as you they would when committing normally to the master branch.

Need a backdoor?

For a lead-dev it may be a good option to provide a backdoor around all the pull-request, review blocking for the case of an emergency. For example during a release you need to adjust something (aka quick-fix while releasing). I know, this will never happen! …but well … yea – somehow those situations di happen – maybe it’s just me 😉

I found that on branch settings one can give right to Exempt from policy enforcement. That’s you backdoor.

Conclusion

I really like what TFS/VSTS has to offer on its GIT integration and how it all plays nicely with work planning, build- and release jobs. With the latest version this stuff is all available directly in the web UI so you don’t need Visual Studio anymore to control TFS. You can use any IDE/TextEditor and the GIT client of your choice. The rest can be done using the web UI. If you did not look at TFS for some time you might wanna give it a try again.

You can create yourself a developer account on Azure and get free VSTS (TFS online) as well as 10 free web apps and more.

Let me know what you think of this stuff.

How to start Unit Testing

Mon, 18 Jul 2016 00:00:00 GMT

So here I put some thoughts and tips how to start more easily with unit-testing. Hopefully I can tear down some walls.

I don’t extend to much on the why as many others have written about it already. Just my personal motivation to write unit-tests:

Besides all the more official pros the one matters for me is that I can sleep better – especially before and after releases – when I know my test-suites are green. I know “hey, the basics are working and I did not break things I once tested. So if something arise it’s probably a border-case issue or at least something within the new stuff so its not high-priority.”.
Second I like to do a good job in design and programming software. Unit-Tests allows me to kind of ensure this the way I can write the tests for the functionality I expect and validate it now and for all times.
Finally I am a developer! I like to spent my time way more on writing code then do stupid repetitively manual testing. Automating things is deep in a Dev’s DNA and that’s a good thing. Write a script/code once – use it many times. That’s how Developers to it! We have to power to do magic things that other can’t do. Its awesome to be Dev – use your power!

Don’t have time to test all?

Easy! I don’t have it either – at least not always. You can start with the most important parts. You will never ever have 100% code coverage. Even 80-90% is hard. And who cares about those values anyway! Don’t get me wrong – I like to run code-coverage from time to time to see what parts of the code are lacking unit-test. But the fixed percentage number can lead to write strange unit-tests just to get another percent code-coverage.

Start small

Give yourself success and start small. When writing your next peace of code think about it: “What is the core logic of this? Let’s write a small unit-tests which ensures this basic functionality.”. Start using baby-steps will give you success experience which is very important to not end frustrated and throw it away again.

You probably will find code you need to refactor to test it. Even after years of unit-testing I find myself refactoring code because of writing the unit-tests showed it is not nicely organized (eg. separated). So writing unit-tests leads to writing better productive code. That’s a real good side-effect of unit-testing / test-automation.

What should I test first?

So where to start with unit-testing? Where should I spend my time first regarding writing automated unit-test? Here is my priority list:

Test core functionality first. For me this normally is the domain-logic / business-logic. This is the hart of the app. Its the part that brings in the real value. If nothing else is validated but this part of your app should be.
Regression bugs: If a bug will be re-introduced in a later release again even if it was fixed already, its called a “regression bug”. These should never happen. It makes customers really unhappy. So if I fix a bug I try to write a unit-test for it do be sure this bug will never ever make its way into a future release.
Extend automated tests as needed and as you can make time available for it.

Important: do start write unit-tests where it make the most sense and adds the most value! They don’t have to be perfect. They don’t need to shine.

How can I reduce the time it needs to write unit-test?

First, it needs practice. You will learn what make testing easier and what doesn’t. Let me introduce you my time-savers:

Write clean-code using SOLID principals – especially useful for writing unit-tests is the “Single Responsibility” principal. This basically means that one class should have only one job. Keep classes small und tire them to a single purpose. This results in clean, maintainable and extensible code and also make unit-testing for those classes a lot easier. Together with “Dependency inversion” principle (DI) this is a real enabler as you don’t need to mock/fake the whole universe.
Don’t forget about your developer skills when writing unit-tests! Use language features like OOP, inheritance, extension methods and handy patterns. You can make base-classes and reusable helper. Cool coding stuff! Don’t do stupid copy-paste just because these are unit-tests. Use you engineering power and write clean code in unit-tests. They get more maintainable and it makes a lot more fun to write unit-tests.
Don’t do automated UI testing until you really, really, really need it. This needs a lot of time and they normally tend to need a lot of maintenance time. Instead use patterns like MVVM and *only* test your view-model logic.

Example:

Let’s image you are writing some migration software which reads from one datasource, transform the data and write it to another datasource.

Instead of writing one class that read, transform and write you should have written tree classes regarding single responsibility: one for reading, one for transforming and one for writing.

You application can chain these classes together using interfaces for them as promoted by dependency inversion.

Now you have tree classes and three interfaces. What do you think will be the most important part to write unit-tests for? Right: the transformation. This is why your app exists at all so make sure this central component works. Its easy as you have interfaces for reading and writing. You can mock or fake them easily and have tests ready.

public interface IMyReader
{
    IEnumerable Read();
}

public interface IMyWriter
{
    Write(string data);
}

public interface IMyTransformer
{
    void Transform();
}

public class MyTransformer : IMyTransformer
{
    public MyTransformer(IMyReader reader, IMyWriter writer)
    {
        ...
    }

    public void Transform()
    {
        foreach (var line in reader.Read())
        {
            // DO TRANSFORM
            writer.Write(line);
        }
    }
}

Now the unit-tests for MyTransformer can be as easy as follows:

var reader = A.Fake();
reader.CallTo(() => Read()).Returns("test input string");

var writer = A.Fake();
reader.CallTo(() => Write()).MustHaveHappened();

var transformer = new Transformer(reader, writer);
transformer.Transform();

// TODO: Additional test of the transformation result

Note: The above code use my famous mock/fake-library “Fake it easy” (best-of Moq and Rhino Mocks).

Libraries

To get you started writing unit-tests for your C# code here the libraries / tools I use for most of my testing:

NUnit – Unit-Testing framework
NUnit Test Adapter – Allows running NUnit tests on VSTS/TFS and within Visual Studio Test-Runner
MSTest – Unit-Testing framework from Microsoft
Fake it easy – Mock and Fake library to setup fake implementations in no time
[FluentAssertion](http://fluent assertion) – Let’s one write the assertions with a nice and fluent syntax
R# Testrunner – To run tests during development

Now go and write unit-tests!

Decouple IoC/DI container when authoring a C# library

Sat, 16 Jul 2016 00:00:00 GMT

Decouple IoC/DI container when authoring a C# library

Intro

Often when writing library code the integration of an IoC/DI container (eg. Unity, SimpleInjector, etc.) gets a topic. My goals are to use Inversion Of Control (IoC) and dependency injection (DI) to decouple things but don’t force the application to use one or another specific container. The user of my libraries should be free in choosing its IoC/DI container. On the other hand I like to give the users of my library a easy way to register the defaults types – again – without forcing one or another IoC/DI container product.

I noticed how ASP.Net core does their DI: they ship their own IoC/DI with the option to replace it with the one of your choice. For my little libraries this feels like an overkill. Even if writing a IoC/DI would be a nice and cool programming task 😉

Solution

So I decided to do it way simpler by introducing a interface for register types with any IoC/DI container. I normally don’t use the Service-Locator pattern in my source-codes but there are very rare situations where it gets hard without it. So I also added an minimalistic interface for service location.

IServiceContainer:

It features three methods for register types with the DI container: RegisterTransient(), RegisterSingleton() and RegisterInstance(). See code and doc below for more details.

/// <summary>
    /// Specify the API of a basic Service Container implementation.
    /// </summary>
    /// <remarks>
    /// All RegisterType-Calls should be implemented against this interface so the
    /// final DI/IoC-Container can be freely choosen if there is a IServiceContainer
    /// implementation for it. If not one can easely write its own.
    /// </remarks>
    public interface IServiceContainer
    {
        /// <summary>
        /// Registers the type <typeparamref name="TFrom"/> to be implemented
        /// by type <typeparamref name="TTo"/> with an optional <paramref name="factory"/>.
        /// TRANSIENT: Each time the type is resolved it will create a new instance.
        /// </summary>
        /// <typeparam name="TFrom">The type of from.</typeparam>
        /// <typeparam name="TTo">The type of to.</typeparam>
        /// <param name="factory">The optional factory expression.</param>
        void RegisterTransient<TFrom, TTo>(Func<TFrom> factory = null) where TTo : TFrom;

        /// <summary>
        /// Registers the type <typeparamref name="TFrom"/> to be implemented
        /// by type <typeparamref name="TTo"/> with an optional <paramref name="factory"/>.
        /// Singleton: A new instance is created once and then used as a Singleton.
        /// </summary>
        /// <typeparam name="TFrom">The type of from.</typeparam>
        /// <typeparam name="TTo">The type of to.</typeparam>
        /// <param name="factory">The optional factory expression.</param>
        void RegisterSingleton<TFrom, TTo>(Func<TFrom> factory = null) where TTo : TFrom;

        /// <summary>
        /// Registers the type <typeparamref name="TFrom"/> to be implemented
        /// by type <typeparamref name="TTo"/> with a pre-instantiated <paramref name="instance"/>.
        /// Instance: The given <paramref name="instance"/> will be used. No new instance is created.
        /// </summary>
        /// <remarks>
        /// Some containers refer this as "external".
        /// </remarks>
        /// <typeparam name="TFrom">The type of from.</typeparam>
        /// <param name="instance">The pre-created instance to use when an instance of <typeparamref name="TFrom"/> is requested.</param>
        void RegisterInstance<TFrom>(object instance);
    }

IServiceLocator:

As mentioned I try not do use the service-locator pattern (why?) but rarely I have to. So I added the IServiceLocatorinterface with the methods Resolve()and ResolveAll().

/// <summary>
    /// Interface for locating/resolving services/types.
    /// </summary>
    /// <remarks>Use this service locator pattern with care. Use constructor parameters instead
    /// whereever possible!</remarks>
    public interface IServiceLocator
    {
        /// <summary>
        /// Resolves a single instance of <typeparamref name="T"/>.
        /// </summary>
        /// <typeparam name="T">The Type one needs a instance of.</typeparam>
        /// <returns>A single instance of <typeparamref name="T"/>.</returns>
        T Resolve<T>();

        /// <summary>
        /// Resolves all instances ot <typeparamref name="T"/>.
        /// </summary>
        /// <typeparam name="T">The Type of the instances to resolve.</typeparam>
        /// <returns>All instances ot <typeparamref name="T"/>.</returns>
        IEnumerable<T> ResolveAll<T>();
    }

You now may ask “Who does the real work?”.

I have an assembly / NuGet package for the concrete implementation of the above interfaces using Microsoft Unity Version 4. Its basically an adapter that routes the interface to the corresponding Unity API:

/// <summary>
    /// Implements <see cref="IServiceContainer"/> and <seealso cref="IServiceLocator"/> using Microsoft Unity. Use this
    /// class if you like to get the isolutions.Library default registreions to
    /// register with a Unity Container.
    /// </summary>
    /// <seealso cref="IServiceContainer" />
    /// <seealso cref="IUnityContainer"/>
    public class UnityServiceContainer : IServiceContainer, IServiceLocator
    {
        private readonly IUnityContainer container;

        /// <summary>
        /// Initializes a new instance of the <see cref="UnityServiceContainer" /> class.
        /// </summary>
        /// <param name="container">The optional Unity container.</param>
        public UnityServiceContainer(IUnityContainer container = null)
        {
            this.container = container ?? new UnityContainer();

            // Register Unity-Implementation
            RegisterSingleton<IServiceLocator, UnityServiceContainer>(() => this);
        }

        /// <inheritDoc/>
        public void RegisterTransient<TFrom, TTo>(Func<TFrom> factory = null) where TTo : TFrom
        {
            CheckContainer();

            if (factory == null)
            {
                container.RegisterType<TFrom, TTo>();
            }
            else
            {
                container.RegisterType<TFrom, TTo>(new InjectionFactory(c => factory()));
            }
        }

        /// <inheritDoc/>
        public void RegisterSingleton<TFrom, TTo>(Func<TFrom> factory = null) where TTo : TFrom
        {
            CheckContainer();

            if (factory == null)
            {
                container.RegisterType<TFrom, TTo>(new ContainerControlledLifetimeManager());
            }
            else
            {
                container.RegisterType<TFrom, TTo>(
                    new ContainerControlledLifetimeManager(),
                    new InjectionFactory(c => factory()));
            }
        }

        /// <inheritDoc/>
        public void RegisterInstance<TFrom>(object instance)
        {
            CheckContainer();

            container.RegisterInstance(typeof(TFrom), instance);
        }

        /// <inheritDoc/>
        public T Resolve<T>()
        {
            CheckContainer();

            return container.Resolve<T>();
        }

        /// <inheritDoc/>
        public IEnumerable<T> ResolveAll<T>()
        {
            CheckContainer();

            return container.ResolveAll<T>();
        }

        private void CheckContainer()
        {
            if (container == null)
            {
                throw new InvalidOperationException("UnityServiceContainer: No container instance configured!");
            }
        }
    }

Usages:

With these interfaces in place the core- as well as my other libraries can give the application service registration helpers like this one:

public static class ServiceRegistration
    {
        public static void UseCore(this IServiceContainer container)
        {
            if (container == null) throw new ArgumentNullException(nameof(container));

            container.RegisterTransient<ISecurityHelper, SecurityHelper>();
            container.RegisterTransient<IUserService, UserService>();
            ...
        }
    }

The application can build up the container instance during startup with code like this very easily and request the libraries to add their default service registrations:

var result = new UnityServiceContainer();
result.UseCore();
result.UseCoreEF6<SampleContext>();
result.UseSampleCore();
...

Conclusion

My core library defines the dependency injection interfaces and all other libraries use only these two interfaces. This means they all are independent from what IoC/DI container is used by the application as long as they get an implementation of these two interfaces.

The Unity implementation is included in the app with a simple NuGet package the app can install. So the app can use my libraries and then decide which of my DI NuGet packages to install. One even can change the DI later on as needed. For example when moving from the full .Net framework to the .Net Core (more on this later).

It’s simple, small and elegant – AND: I didn’t implement my own IoC/DI 🙂

Restructuring of my personal websites and blogs

Fri, 15 Jul 2016 00:00:00 GMT

Restructuring of my personal websites and blogs

After some years with the old websites and blogs several issues arised. For my personal content I had basically two sites:

idev.ch – Hosted my old tech blog as well as some aged software I wrote many years ago.
marcduerst.com – Site hosts my diving- and photoblog.

They both where built using Drupal 7 and hosted on a local hoster.

The issues I had with them:

Security updates of Drupal and – even worst – ever plugin I used to build the sites. I could have installed security updates nearly every week if I wish.
idev.ch isn’t really the name I like to promote my tech blogs. It’s the label the existing users of the old software now.
I like to start over again with my personal tech blog and hopefully keep the drive longer then before and update it more regularly.
Adding posts to the tech blog was way to complicate. I like to spend time on the content; not the the tech behind the blog.

Other goals and requirements:

Minimize time spent on technical blog-maintenance but don’t get lazy on security.
Keep tech blog separated from my dive & photo blog.
Keep up to date with state of the art blogging features. Eg. syndicate, share, monitor/stats, editing
Use a abstract syntax to write technical blog-posts. Markdown would be nice.
Have syntax-highlighting for code-blocks and inline-code.
Use the domain marcduerst.com for my primary personal stuff.
Keep the domain idev.ch for the old software.
Increase priority of tech topics over the dive & photo topics.
Keep dive & photo blog and also update it the future.

Platform evaluation

The main alternatives I came up regarding the tech blog platform where:

Install a software with easier/automated maintanance then my custom Drupal and host the site on my local hoster.
Write my own blog software as an exercise for the new ASP.Net Core platform and Azure Websites.
Use Jekyll software to generate static website based on a GIT repository and it on local hoster or on GitHub pages.
Use a cloud hosted SaaS like wordpress.com.

Each has its pros and cons.

Install a better updateable software may be a minor improvement and needs time to learn and build it.

Write my own needs a lot of time and blogs are not the best learning example anyway. Furthermore I have to implement all current and upcoming features myself so I again spend a lot of time for tech instead content.

Jekyll static site generator based on GitHub is geeky but also has quite a learning curve and needs time to build. New blog features in the future may be built myself.

SaaS (WordPress.com etc) limit to what that platform actually can.

Decision

After I tried all of them I decided go with SaaS as I am set up quickly and has a good and up-to-date featureset for blogging. Furthermore I most not spend any time over the next years to keep the platform running.

Action plan

Based on the above decision and requirements I came up with the following action plan.

Set up a WordPress.com blog => done
Move the existing content from marcduerst.con over to tauchfotos.ch => done
Point marcduerst.com to the new WordPress blog => done
Rebuild idev.ch as a static HTML page just used for the old software I wrote
Maybe set up an additional WordPress blog for the dive & photo content over at tauchfotos.ch. Not yet sure about this.

As you see my action plan makes pretty good progress but there is still work left.

Build a SignalR connection indicator with TypeScript and Knockout

Tue, 24 Nov 2015 00:00:00 GMT

Build a SignalR Connection Indicator with TypeScript and Knockout

I’ve build a message infrastructure for the company I am working for which allow to collect tons of production data from our factory (about >4’000 db records per minute). In addition I’ve built a web frontend to query this data and to live monitor some of the data. The monitoring includes pushing current server throughput statistic data and event messages like error etc. that that happens in the factory next door including lots of additional data to understand what’s behind these errors.

For this monitoring UI I used TypeScript for most of the Javascript part and SignalR to push the data from the server to the web-client. If no new problems (aka errors, warnings etc.) are shown the question was: “Is SignalR just disconnected or are we at the lucky side of the day and don’t have new problems happening in our old software?”. So I decided to implement a little indicator showing the SignalR connection status. As I googled around a little and mixed/tuned/changed existing solutions I thought it would be nice to write down how I did it for later records.

First lets start up with the tech stack I used: ASP.Net MVC 5 w. Razor, TypeScript 1.6, SignalR 2.2.0, Knockout 3.3 and Bootstrap 3.3.5 are the relevant parts here.

Razor View

I’ve build the following HTML (Razor but doesn’t matter here) view that holds the Bootstrap markup for the four connections states.

<div id="connectionStateContainer" class="hidden">
    <span>Status:</span>
    <span class="label label-info">Connecting</span>
    <span class="label label-success">Connected</span>
    <span class="label label-warning">Reconnecting</span>
    <span class="label label-danger">Disconnected</span>
</div>

As you see I used Bootstrap Labels for visualizing the different connection states. Further more I added data-bind attributes that allow Knockout to bind the visible attribute of the labels to the connectionState observable properties on my view-model. The values $.signalR.connectionState.* are from the SingalR Javascript library. The sourrounding DIV #connectionStateContainer has the Bootstrap class ‘hidden’ so it is initially hidden until the view-model is correctly bound. Otherwise I would see all the label for a short time on screen before there are bound and therefore hidden.

Javascript Code

I have a little piece of plain Javascript code within the page to mainly get the TypeScript generated Javascript library firing.

var liveTickerViewModel = new KaLog.LiveTickerViewModel();
ko.applyBindings(liveTickerViewModel, $("#liveMessageTickerRoot").get(0));
$("#connectionStateContainer").removeClass("hidden");
liveTickerViewModel.startSignalRHub($.connection);

As my TypeScript code does not have any view-code (id’s, selectors, etc.) in it I decided to have those four lines of code within the view/page itself so things are separated. It’s a matter of taste / programming-style.

But anyway: the above code first instantiate my little view-model built with TypeScript (see below) then applies the Knockout databinding’s to that part of the page and remove the hidden class from the surrounding container as now the right label should be ‘visible’. Removing the hidden class will actually result in showing the surrounding DIV container including its content.

Last but not least we tell our view-model to start the SignalR Hub.

TypeScript Code

Now the last part that makes it all work, the TypeScript view-model. Below you see the interesting parts of my view-model or let’s say the connection status indicator related parts of the code.

namespace KaLog {
 export class LiveTickerViewModel {

 /**
 * Represent the SignalR connection state.
 */
 public connectionState: KnockoutObservable<number> = ko.observable(4);

 /**
 * Initialize and start the SignalR hub.
 * @param connection
 * @returns {}
 */
 public startSignalRHub(connection: any) {
 var serverMonitoringHub = connection.serverMonitoringHub;
 ...
 connection.hub.stateChanged((state: SignalRStateChange) => this.connectionStateChanged(state));
 connection.hub.start();
 }

 /**
 * Callback method triggered when the connection state of the SinalR connection changes.
 * @param state
 * @returns {}
 */
 public connectionStateChanged(state: SignalRStateChange): void {
 var stateConversion = { 0: 'connecting', 1: 'connected', 2: 'reconnecting', 4: 'disconnected' };
 console.log('SignalR state changed from: ' + stateConversion[state.oldState] + ' to: ' + stateConversion[state.newState]);

 this.connectionState(state.newState);
 }
 }
}

First we declare our view-model class LiveTickerViewModel. Then we declare and initialize the property connectionState which is a Knockout Observable which holds a value of type number. We initialize it to disconnected.

In the startSingalRHub() method we retrieve the SignalR client hub (generated by the SignalR Javascript Client). On the Hub we attach an event-handler for the stateChanged event which calls the connectionStateChanged() method.

The connectionStateChanged() method does some console output (for debugging purpose) and then set the value of the observable property connectionState to the new SingalR connection state. Setting this value will trigger the databinding from Knockout which result in updating the visible attributes of our Bootstrap labels. Done.

If you don’t need the debug console output you can inline the this.connectionState(state) call straight into the connection.hub.stateChanged-lambda and remove the entire connectionStateChanged-method.

Configure IIS for passing static file requests to ASP.NET MVC

Thu, 10 Jul 2014 00:00:00 GMT

Configure IIS for passing static-file requests to ASP.Net/MVC

At my company we had several ASP.Net MVC projects what needed to serve some files with a custom MVC Controller/Action. The general problem with this is that IIS tries hard to serve simple files like PDF’s, pictures etc. with its static-file handler which is generally fine but not for files or lets say file-content served by our own action.

The goal is to switch off the static-file handling of IIS for some paths. One of the current projects came up with the following requirements so I did some research and how we can do this better then we did in past projects.

Requirements:

Switch it off only for /Data/…
Switch it off for ALL file-types as we don’t yet know what files the authors will store in somewhere else.

This means that the default static-file handling of IIS must be switched off by some “magic” IIS config. In other apps we switched it off on a per file-type basis for the entire application. I finally came up with the following IIS-config (in web.config). It sets up a local configuration for the “data”-location only. Then I used a simple “*” wild-card as the path (yes, this is possible) to transfer requests to the ASP.Net. It looks like this:

<location path="data">
  <system.webServer>
    <handlers>
      <add name="nostaticfile" path="*" verb="GET" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
    </handlers>
  </system.webServer>
</location>

Tested with IIS 7.5.

Alternative:
Instead a controller one could also use a custom HttpHandler for serving such special URL’s/Resources. In this project I decided using an action for this because of the central custom security which I needed for the /Data/… requests as well and got for free when using Action instead a HttpHandler.

ASP.NET MVC Areas

Wed, 09 Apr 2014 00:00:00 GMT

ASP.Net MVC Areas

After some initial skepticism regarding Areas, I now use them more and more when building new Web-Applications using ASP.Net MVC. Therefore, I decided to cover some of my thoughts and experiences in a blog post so others may get some inspiration out of it.

Before we start, here’s a link to a general introduction to the area feature of MVC. Check out this article if you are not yet familiar with Areas.

Furthermore, this topic is based on MVC 5 and C# 4 but may also apply to older versions too as Areas are not really a new thing and were first introduced with MVC 2.

Introduction

Areas are intended to structure an MVC web application. Let’s say you’re building a line-of-business application. You may want to separate your modules on one hand from each other and on the other hand from central pieces of your web application like layouts, HTML helpers, etc.

Therefore an area should be pretty much self-contained and should not have too much interaction with other areas—as little as possible—as otherwise the dependencies between the modules and their implementation grows more and more, undercutting separation, and resulting in less maintainability.

How one draw the borders of each area depends on the need of the application and company. For example, modules can be separated by functionality or by developer/developer-team. In our line-of-business application, we may have an area for “Customer Management”, one for “Order Entry”, one for “Bookkeeping” and one for “E-Banking” as they have largely separate functionality and will likely be built by different developers or even teams.

The nice thing about areas—besides modularization/organization of the app—is that they are built in MVC and therefore are supported by most tools like Visual Studio, R# and most libraries. On the negative side, I can count the lack of good support in the standard HtmlHelpers as one need to specify the area as a routing-object property and use the name of the area as a string. There is no dedicated parameter for the area. But to put that downside into perspective, this is only needed when making an URL to another area than the current one.

Application Modularization using Areas

In my point of view, modularization using areas has two major advantages. The first one is the separation from other parts of the application and the second is the fact that area-related files are closer together in the Solution Explorer.

The separation—apart from the usual separation advantages—is helpful for reviews as the reviewer can easily see what has changed within the area and what changed on the core level of the application and therefore needs an even-closer look. Another point of the separation is that for larger applications and teams it results in fewer merge conflicts when pushing to the central code repository as each team has its own playground. Last but not least, its nice for me as an application-developer because I know that when I make changes to my area only I will not break other developers’ work.

As I am someone who uses the Solution Explorer a lot, I like the fact that with areas I normally have to scroll and search less and have a good overview of the folder- and file-tree of the feature-set I am currently working on. This happens because I moved all area-related stuff into the area itself and leave the general libraries, layouts, base-classes and helpers outside. This results in a less cluttered folder-tree for my areas, where I normally spend the majority of my time developing new features.

Tips and Tricks

Move all files related to the area into the area itself including style-sheets (CSS, LESS, SASS) and client-side scripts (Javascript, TypeScript).
Configure bundles for your area or even for single pages within your area in the area itself. I do this in an enhanced area-registration file.
Enhance the default area registration to configure more aspects of your area.
When generating links in global views/layouts use add the `area=“” routing attribute so the URL always points to the central stuff instead being area-relative.

For example: if your application uses @Html.ActionLink() in your global `_layout.cshtml?, use:

@Html.ActionLink(&amp;quot;Go to home&amp;quot;, &amp;quot;Index&amp;quot;, &amp;quot;Home&amp;quot;, new { area = &amp;quot;&amp;quot; });

Area Registration / Start Up

And here is a sample of one of my application’s area registrations:

public class BookkeepingAreaRegistration : AreaRegistration
{
  public override string AreaName
  {
    get { return &amp;quot;Bookkeeping&amp;quot;; }
  }

  public override void RegisterArea(AreaRegistrationContext context)
  {
    RegisterRoutes(context);
    RegisterBundles(BundleTable.Bundles);
  }

  private void RegisterRoutes(AreaRegistrationContext context)
  {
    if (context == null) { throw new ArgumentNullException(&amp;quot;context&amp;quot;); }

    context.MapRoute(
      &amp;quot;Bookkeeping_default&amp;quot;,
      &amp;quot;Bookkeeping/{controller}/{action}/{id}&amp;quot;,
      new { controller = &amp;quot;Home&amp;quot;, action = &amp;quot;Index&amp;quot;, id = UrlParameter.Optional }
    );
  }

  private void RegisterBundles(BundleCollection bundles)
    {
      if (bundles == null) { throw new ArgumentNullException(&amp;quot;bundles&amp;quot;); }

      // Bookings Bundles
      bundles.Add(new ScriptBundle(&amp;quot;~/bundles/bookkeeping/booking&amp;quot;)
          .Include(&amp;quot;~/Areas/bookkeeping/Scripts/booking.js&amp;quot;
      ));

      bundles.Add(new StyleBundle(&amp;quot;~/bookkeeping/css/booking&amp;quot;)
          .Include(&amp;quot;~/Areas/bookkeeping/Content/booking.css&amp;quot;));

      // Account Overview Bundle
      …
  }
}

As you can see in this example, I enhanced the area registration a little so area-specific bundles are registered in the area-registration too. I try to place all areas-specific start-up code in here.

Folder Structure

As I wrote in one of the tips, I strongly recommend storing all area-related files within the area’s folder. This includes style-sheets, client-side scripts (JS, TypeScript), content, controller, views, view-models, view-model builders, HTML Helpers, etc. My goal is to make an area self-contained so all work can be done within the area’s folder.

So the folder structure of my MVC Apps look something like this:

App_Start
Areas
- Content
- Controllers
- Core
- Models
- Builders
- Scripts
- Views
bin
Content
Controllers
HtmlHelpers
Models
- Builders
Scripts
Views

As you can see, each area is something like a mini-application within the main application.

Add-ons using areas deployed as NuGet packages

Beside structuring an entire MVC Application, another nice usage is for hosting “add-on’s” in their own area.

For example, lately I wrote a web-user-interface for the database-schema migration of our meta-data framework Quino. Instead of pushing it all in old-school web-resources and do binary deployment of them, I built it as an area. This area I’ve packed into a NuGet package (.nupkg) and published it to our local NuGet repo.

Applications which want to use the web-based schema-migration UI can just install that package using the NuGet UI or console. The package will then add the area with all the sources I wrote and because the area-registration is called by MVC automatically, it’s ready to go without any manual action required. If I publish an update to the NuGet package applications can get these as usual with NuGet. A nice side-effect of this deployment is that the web application contains all the sources so developers can have a look at it if they like. It doesn’t just include some binary files. Another nice thing is that the add-on can define its own bundles which get hosted the same way as the MVC app does its own bundles. No fancy web-resources and custom bundling and minification is needed.

To keep conflicts to a minimum with such add-on areas, the name should be unique and the area should be self-contained as written above.

Conclusion

Areas are a very nice thing in ASP.Net one can use for modularisation and therefore separation of building block of your web-application. They have some issues that could be improved (like the lack of typed support in the standard helpers) but they are definitivelly worth taking a closer look at. For my web-applications I use them mostly right from the beginning or as soon as I see that this will be a not trivial web-application or if multiple developers work on them. The add-on’s part described in the above section will be used much more for our central / common features that are available to multiple web-applications.

marcduerst.com — Notes on Code & Cloud

AI-Assisted Software Product Development — A Practical Field Report

Spec-Driven Development is Back

Build a Machine-Readable Briefing

A Folder Structure That Works

Make the AI Assistant Yours

My Refinement and Implementation Process

A Note on Code Reviews

Beyond Coding: AI in the Product Lifecycle

AI-Assisted Market Analytics

AI-Assisted Security Reviews

AI-Assisted Operations Debugging

AI-Assisted Cloud Cost Debugging

More Ideas for AI Assistance

Thoughts on What This Means

The Industry Can’t Ignore This

Quality Still Requires Expertise

The Junior Developer Question

Organizational Change Beyond Code

The Pace and the Uncertainty

The Environmental Cost

Integrating Machine Learning — A Software Engineer's Guide

What is a Data Scientist

Data Analysts

Data Scientists

How Data Science and Engineering Work Together

Phase 1: Data Science Research

Phase 2: Model Training

Phase 3: Software Engineering Integration

Keys to Good Collaboration

Offline vs. Online: The Most Important Distinction

Offline (Batch Predictions)

Online (Near Real-Time)

Decision Framework: Offline or Online?

Tips for Online ML

Drift vs. Skew — Know the Difference

What Does “Quantize a Model” mean?

Closing Thoughts

Tech stack for hosting small apps on Azure for free

Summary

My requirements

Choose tech-stack

Azure Storage Accounts

Azure Static Web Apps

Limitations

Resulting Azure Resource Group

Implementation

Secure access to data

Static web app

CORS (Browser Security)

CI / CD with GitHub Actions

Alternatives

Closing

Monitor a pod's container resources in GCP

The problem

The solution

Summary

Scale and fix a Azure Service Fabric cluster

Situation

Learning: Get a healthy cluster after each operation

Learning: How to scale-out/-in manually

Learning: Never ever go below 3 nodes

Learning: Seed nodes promotion

Learning: Node management in SFX with “Advanced Mode”

Learning: SF will not re-balance fault-domain if you scale manually

Learning: Use supported VM-sizes

Conclusion

Order Azure Log Analytics result columns

Problem

Solution

Track SQL data with Azure alerts and dashboard charts

Goal

Solution

Query SQL database using a Azure Logicapp

Show results of SQL query on Azure dashboard

Solution

Azure DevOps: Retrieve Application Insights instrumentation key

The Problem

The Solution

What’s Different