Patchwork MIPS: dma: if BMIPS5000, flush region just like r10000

login
register
mail settings
Submitter Jim Quinlan
Date Aug. 27, 2013, 8:57 p.m.
Message ID <1377637071-32740-1-git-send-email-jim2101024@gmail.com>
Download mbox | patch
Permalink /patch/5776/
State Accepted
Delegated to: Ralf Baechle
Headers show

Comments

Jim Quinlan - Aug. 27, 2013, 8:57 p.m.
The BMIPS5000 (Zephyr) processor utilizes instruction speculation. A
stale misprediction address in either the JTB or the CRS may trigger
a prefetch inside a region that is currently being used by a DMA
engine, which is not IO-coherent.  This prefetch will fetch a line
into the scache, and that line will soon become stale (ie wrong)
during/after the DMA.  Mayhem ensues.

In dma-default.c, the r10000 is handled as a special case in the
same way that we want to handle Zephyr.  So we generalize the
exception cases into a function, and include Zephyr as one
of the processors that needs this special care.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 arch/mips/mm/dma-default.c |   16 ++++++++++------
 1 files changed, 10 insertions(+), 6 deletions(-)
root - Sept. 4, 2013, 4:53 p.m.
Dan,

On Tue, Aug 27, 2013 at 04:57:51PM -0400, Jim Quinlan wrote:
> Date:   Tue, 27 Aug 2013 16:57:51 -0400
> From: Jim Quinlan <jim2101024@gmail.com>
> To: ralf@linux-mips.org, linux-mips@linux-mips.org
> cc: cernekee@gmail.com, Jim Quinlan <jim2101024@gmail.com>
> Subject: [PATCH] MIPS: dma: if BMIPS5000, flush region just like r10000
> Content-Type: text/plain
> 
> The BMIPS5000 (Zephyr) processor utilizes instruction speculation. A
> stale misprediction address in either the JTB or the CRS may trigger
> a prefetch inside a region that is currently being used by a DMA
> engine, which is not IO-coherent.  This prefetch will fetch a line
> into the scache, and that line will soon become stale (ie wrong)
> during/after the DMA.  Mayhem ensues.
> 
> In dma-default.c, the r10000 is handled as a special case in the
> same way that we want to handle Zephyr.  So we generalize the
> exception cases into a function, and include Zephyr as one
> of the processors that needs this special care.

Is this a processor erratum or just documented, undesireable behaviour?

In case of the R10000 family it's the later and it also only affects
systems without cache coherency.  In such systems it is also possible
that cachelines in speculative-dirty state will be created by a
speculativly executed store instruction.  This is normal - but on a
cache coherent system the coherency logic would prevent such speculativly
dirty lines from being written back to memory.

To avoid this from happening non-coherent R10000 systems also require their
kernel to be built with a special compiler option that inserts cache barrier
operations wherever a speculativly dirty line otherwise might be created.

Patch is looking good.

  Ralf

Patch

diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index aaccf1c..468f7f9 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -50,16 +50,20 @@  static inline struct page *dma_addr_to_page(struct device *dev,
 }
 
 /*
+ * The affected CPUs below in 'cpu_needs_post_dma_flush()' can
+ * speculatively fill random cachelines with stale data at any time,
+ * requiring an extra flush post-DMA.
+ *
  * Warning on the terminology - Linux calls an uncached area coherent;
  * MIPS terminology calls memory areas with hardware maintained coherency
  * coherent.
  */
-
-static inline int cpu_is_noncoherent_r10000(struct device *dev)
+static inline int cpu_needs_post_dma_flush(struct device *dev)
 {
 	return !plat_device_is_coherent(dev) &&
 	       (current_cpu_type() == CPU_R10000 ||
-	       current_cpu_type() == CPU_R12000);
+		current_cpu_type() == CPU_R12000 ||
+		current_cpu_type() == CPU_BMIPS5000);
 }
 
 static gfp_t massage_gfp_flags(const struct device *dev, gfp_t gfp)
@@ -230,7 +234,7 @@  static inline void __dma_sync(struct page *page,
 static void mips_dma_unmap_page(struct device *dev, dma_addr_t dma_addr,
 	size_t size, enum dma_data_direction direction, struct dma_attrs *attrs)
 {
-	if (cpu_is_noncoherent_r10000(dev))
+	if (cpu_needs_post_dma_flush(dev))
 		__dma_sync(dma_addr_to_page(dev, dma_addr),
 			   dma_addr & ~PAGE_MASK, size, direction);
 
@@ -284,7 +288,7 @@  static void mips_dma_unmap_sg(struct device *dev, struct scatterlist *sg,
 static void mips_dma_sync_single_for_cpu(struct device *dev,
 	dma_addr_t dma_handle, size_t size, enum dma_data_direction direction)
 {
-	if (cpu_is_noncoherent_r10000(dev))
+	if (cpu_needs_post_dma_flush(dev))
 		__dma_sync(dma_addr_to_page(dev, dma_handle),
 			   dma_handle & ~PAGE_MASK, size, direction);
 }
@@ -305,7 +309,7 @@  static void mips_dma_sync_sg_for_cpu(struct device *dev,
 
 	/* Make sure that gcc doesn't leave the empty loop body.  */
 	for (i = 0; i < nelems; i++, sg++) {
-		if (cpu_is_noncoherent_r10000(dev))
+		if (cpu_needs_post_dma_flush(dev))
 			__dma_sync(sg_page(sg), sg->offset, sg->length,
 				   direction);
 	}