LiPo Cascade Failure (Dept of Cautionary Tales)

Here’s the short version of what happened to my pack recently.  I frankly had no idea this was possible, yet, from the response of a few friends with considerable lipo experience, it is painfully possible, if not common.

The bottom line.  One bad cell can cause a small cascade failure of a pack.  That failure can cause the larger cascade of any packs paralleled to it.  If allowed to progress, it can bury your entire pack.

Here’s how my pack is wired.

I just happened to notice my pack voltage was extremely low one day, while the bike was sitting in the garage, and started charging it immediately.  Three of the banks of paralleled packs responded normally.  The fourth was kind of funny, and the fifth kept dumping voltage.  It would charge, then immediately drop down to about 2V per cell (group).  I pulled the pack apart, and charged each individual pack.  All the packs were fine, took charges normally, and held their charge, except for the 4 packs in that one wonky bank.  They were all toast.

What happened?  It would seem that I had a cell failure in one of the packs in the wonky bank.  Because paralleled cells want to balance normally, when that cell shorted, all the other cells started discharging into it.  They then dropped below salvageable voltage, and the packs were toast.  At that point, the rest of the pack started discharging into that bad bank, and I came close to losing the whole 74V 20Ah pack.

How to keep this from happening?  Well, testing the packs, for one…  running them through cycles, making sure they all work normally, and, I’d add a step I didn’t before, testing them and cycling them when they’re paralleled in banks.  Also, test and match the internal resistance of each pack with it’s buddies in it’s bank.

Part two.  Get a cell-level monitor.  Either a fancy cell-log (only $30 or so from Hobby King) or a simple one (at $6 each), but something to keep an active eye on each bank of cells, and alarm you if you have a problem.

The end result?

I pulled the bad packs, in the bad bank, out.  I rebuilt the pack using 3, rather than 4 packs in parallel, giving me a 15Ah 74V pack, which I’m dubbing the “MicroPack”.  I’m adding cell loggers.

I can’t just add new 4s packs to build it back up to the same Ah rating.  They won’t have the same internal resistance, and will be over-taxed as a result.  I can either start over with all-new packs, or live with one smaller capacity pack with what I have left.  I can’t build a second “MicroPack”, either, and put it in parallel with my first pack, again, because it won’t match.

The good news is, with the MicroPack the bike still is scalded-cat quick…  with a little more voltage sag than I’d like, but still fun.  …but a very expensive lesson learned.

Update:  I’m not at all convinced that the bad bank caused the discharge of the entire pack.  Through some discussion on ElMoto, I’m reminded that the controller was on, though the contactor was not…  so there had to be some current draw.  If that’s the case, the pack voltage dropped, probably causing the bad bank to fail.  Stay tuned.

Care and Feeding of LiPo

I have been wondering about storing my bike outside in the cold New England winters.  Here’s a great post that talks about that, and other storage, temperature and charging issues with lipo batteries from none other than the RC aviation community.  Who else?

He focuses on getting the most life out of lipo, and the secret is in not abusing them…  knowingly or not.

Lithium lifetime is based on several factors:

1. Temperature while in use
2. Temperature while in storage (for our purposes, “storage” is the same as “maintaining a constant voltage level”)
3. Percentage of “full charge” (4.20v/cell) to which a cell is charged
4. Percentage of “full discharge” (depending on chemistry, somewhere between 2.5v and 3.2v) reached by a cell
5. Rate of charge (depending on chemistry, between 0.3C and 3C may not be excessive)
6. Rate of discharge (depending on chemistry, up to 20C may not be excessive)
7. Charge level while in storage

In particular, there’s the issue of charging when the batteries are cold. And by cold, he means around 32F.  If you top out the battery at that temperature, you’ve overcharged it.  When it gets warmer, your 4.2V charge is going to go well over that, apparently, so you may have taken as much as 10% off the life of the pack.

Here’s another very informative post on lipo use in general: Lipo Usage Best Practices.

Finally, none other than Battery University:

Many battery users are unaware that consumer-grade lithium-ion batteries cannot be charged below 0°C (32°F). Although the pack appears to be charging normally, plating of metallic lithium can occur on the anode during a subfreezing charge. The plating is permanent and cannot be removed with cycling. Batteries with lithium plating are known to be more vulnerable to failure if exposed to vibration or other stressful conditions. Advanced chargers, such as those made by Cadex, prevent charging Li-ion below freezing.

The Lipo Project: Balance Block and the Mini Bus Bullet Block

As is often the case when I’m in the middle of a project I put all this stuff together and kind of forgot to post it here.  Sorry.

Here’s what I ended up with for connections and wire management for the lipo packs.

First, with the help of David O’Brien, I got these little things together for paralleling the balance tabs:

After we got done, he found some boards that were set up as strips in the bottom of some drawer, but watching him solder this thing was like watching Leonardo paint.

Here’s what I came up with to parallel the 9mm bullet connectors on the Turnigy packs:

A simple chunk of copper, drilled with holes to accept the connectors.  Add a little heatshrink and you gotcha Mini Bullet Bus Block.

Connectors working loose is a huge issue, so I wanted to figure out some way to mount them, lock them, and help manage the cables.  I made up a simple ABS plate with some cutouts and ridges that does the trick.

Here’s what it all looks like in the bike:

The LiPo Project: Rethinking the Harness

Slow day at work yesterday…  did a little playing around with the harness to try to get less of a mess:

XBox 360 PowerBrick Hack Supply for Turnigy Charger (FTW)

Simple as pie.  XBox 360 PowerBrick from eBay- $14.  Cut connector, jump the red and blue wires (turns on the power supply), and your black and yellow wires deliver 12.5VDC at around 16amps.

The LiPo Project- Wrapup

Here are the harnesses- each one is crimped in the middle, paralleling the four packs in each module, then series connected to the next module:

There’s one connector that’s flagged for the charger in/out for each module in yellow.

Here it is on the bike:

The charging connections, including the balance tabs that are paralleled within the modules, are bundled to avoid any confusion when charging.

Preliminary results of running it around the neighborhood?  SICK!

Dept of Watching Grass Grow: Testing the Turnigy, Part Two

So here’s the final strategy.  (Comment from RC” “ur doin it rong.  but not too rong”

The idea is to discharge the packs down to a safe level, (see additions below) so starting with topping up the charge is kind of a waste of time.  Instead, I decided to just hit them right off with a discharge for two hours…  which, coincidentally, brings them down to around 3.1-2.9V.  Their safe minimum.  Right away, 2 1/2 hours saved.

So, at the end of that, starting usually at around 3.8V, we’re ending up with, for example, cell voltages of 3.26 – 3.14 – 3.07 – 3.06.  Not bad.  And, they balance out to within .01V by the time they’re back up to 3.5V. Total elapsed time is about two hours for the discharge, and about an hour for the charge.

One comment.  One thing I learned early on with digital cameras, you’ve got to label (and number) your batteries.  There’s always one battery in a pack that’s bad, and, simple as it seems, if you don’t label them you have no way to track their performance, especially over their lifetime.  Simple housekeeping…

Update: OK, here’s where I’m at.

The goal here is to weed out bad packs.  The other goal, which I can’t do right now, is to try to match up packs based on the internal resistance of each cell, making everything more balanced.  This is something I think I’ll tackle later on, once I have more chargers and more packs.  For now, I just want to flag any packs with cells that are weak, which should show up as low voltage after a bit of a discharge, and uneven voltage after a top-off charge.

Bottom line, I’m continuing with the process.  After I run the bike gently a few times I’ll do balance charges of each module and see how that looks.  I’m going to save the fine-tuning for when RC is here…  and I can ply him with beer.

Dept of Watching Grass Grow: Testing the Turnigy

The primary purpose of testing the Turnigy lipo packs, at least so I’ve been told, is to determine how each cell “drops off”, or, how the voltage decreases as the cell discharges, near the minimum charge level.  If a cell gets to a point and drops off too rapidly, you have a bad cell, and thus a bad pack.  That can make it build up heat, which, especially for lipo, is a very Bad Thing.

The charger I’m running is the Turnigy 150W unit shown above.  I’m setting it up to discharge the packs, and it’s showing a 5amp discharge setting but is averaging a little over 1.5amps actual discharge.  For a fully charged pack it takes about 2 hours to go from 4.18V to 3.7, or so, and then it hits the time limit.  I’ve hit it for another discharge cycle, and we’ll see where that brings my voltage, but ideally I’d like to get it down to nearly the safe minimum- probably around 3-3.2V.

I’d really like to get this all done by August 14th, for a show I’d like to run it in…  but we’ll see.  This is one process I don’t think it’s safe to rush.

On another note, the power supply I ordered was for a computer peripheral, at around $10.  Rated at 12V and 6amps, I thought it would do the trick, but no.  After a few minutes it just cuts out.  Right now I’m just running the charger off of my big scooter batteries and it’s going fine, but that’s definitely something that I need to tend to.

Connecting the Charger (Turnigy LiPo)

You know the old saying about a picture being worth a thousand words and all…  well, I seem to have missed the words, “Connect the Main Output leads to the power leads of the battery”.  No seriously.  I’ve been thinking the battery charged entirely through the balance connectors…  now, of course, I realize that’s kind of silly, considering how small they are…

So here it is, the main power input on the right (just to confuse you, right next to the balance connections) and the main charging output coming out of the left side.  You can get a look at the display here too.

OK.  Now.  Here’s the new wiring plan, based on my newfound Wisdom.

The LiPo Project… The Connector Strategy

I’ve decided how I’m going to wire these babies up.  Because the Turnigy charger is so cool, and reads out the cell status for each individual cell, I’m not going to be cutting and hard-wiring the packs together because I want to be able to pull individual packs out and let the charger balance them one at a time.  So.

On the balance leads, I’m setting up a circuit board using a simple test board to hold 4 of these suckers.  They are from Digikey, the JST B5B-XH-A.

Then I’m going to make the power feed pigtails using these slick bullet connectors from HobbyKing.  Reports are that they’re a total PIA to assemble, but they’re awesome, so I’m going to give it a shot.  They can be found here, at HobbyKing. 

Once I have all five modules together, I”ve ordered some 10″ wide shrink tubing to hold each module together.

The catch is, I have to go back to the battery mount on the bike and make room for one more module.  These are only slightly smaller than the 6s packs I was planning for, so 16 fit perfectly.  20, not so much.

The 10″ wide shrink tube I got from AllBattery.com, along with the little JST-XH 5-conductor pigtails that are going to the PC board.